Ace Cloud Interviews
Home/AWS Tutorial/EKS Anywhere
🐳

AWS Containers

EKS Anywhere

Run EKS on your own on-premises infrastructure or VMware vSphere

EKS Anywhere is a deployment option for Amazon EKS that lets you run Kubernetes clusters on your own on-premises infrastructure, VMware vSphere, bare metal, Nutanix, Apache CloudStack, or Snow devices - using the same EKS tooling and APIs as cloud EKS. It gives organizations with data sovereignty, latency, or compliance requirements the ability to run Kubernetes with AWS-supported tools and optionally connect to AWS services. EKS Anywhere is distinct from EKS on Outposts - you fully own and manage the infrastructure while AWS provides the tooling.

EKS Anywhere vs EKS vs Outposts - Understanding the Options

AWS offers multiple ways to run Kubernetes. Choosing the right one requires understanding the infrastructure ownership model, connectivity requirements, and what AWS manages.

OptionInfrastructure OwnerControl Plane LocationAWS Connection RequiredUse Case
EKS (Cloud)AWSAWS (fully managed)Yes - you are in AWSStandard cloud workloads
EKS on Outposts (Extended)AWS (Outposts rack)AWS region (remote worker nodes on Outposts)Yes - requires AWS region connectivityLow-latency apps needing AWS-managed control plane
EKS on Outposts (Local)AWS (Outposts rack)On-premises (control plane on Outposts)No - fully offline capableDisconnected or strictly local compliance
EKS Anywhere (vSphere/bare metal)YouOn-premises (you manage nodes)Optional - airgapped supportedData sovereignty, existing on-prem infra
EKS DistroYouYour infrastructureNo - just the K8s distributionDIY - just want the EKS K8s build
💡

EKS Distro is just the Kubernetes distribution that EKS uses - upstream K8s with AWS patches and security fixes. EKS Anywhere is the full tooling layer (cluster lifecycle management, add-ons) built on top of EKS Distro. You get EKS Distro for free; EKS Anywhere has an optional support subscription.

Infrastructure Providers and Hardware Requirements

EKS Anywhere supports multiple infrastructure providers. Each has different setup requirements and feature parity levels.

ProviderStatusMin HardwareNotes
VMware vSphereGAvSphere 7.0+, 2 vCPU/8GB RAM per nodeMost mature - full feature support
Bare MetalGAx86_64 servers, Tinkerbell for PXEUses Tinkerbell for provisioning
NutanixGANutanix AOS 6.5+Full lifecycle management
Apache CloudStackGACloudStack 4.14+Community-supported provider
AWS SnowGASnowball Edge, Snow FamilyFor disconnected/edge AWS devices
Docker (local)Development onlyDocker Desktop on Mac/LinuxNot for production - dev/test only

Minimum cluster requirements: 1 admin machine (Linux/Mac), 1 control plane node, 1 worker node. For HA production: 3 control plane nodes + 3+ worker nodes.

bash
# Install eksctl-anywhere CLI
brew install aws/tap/eks-anywhere

# Generate cluster configuration
export CLUSTER_NAME=my-eks-anywhere
export VSPHERE_DATACENTER=dc-01
export VSPHERE_DATASTORE=datastore-01
export VSPHERE_FOLDER=/dc-01/vm/eks-anywhere
export VSPHERE_NETWORK="VM Network"
export VSPHERE_SERVER=vcenter.example.com

eksctl anywhere generate clusterconfig $CLUSTER_NAME \
  --provider vsphere > cluster.yaml

# Review and edit cluster.yaml, then create
export EKSA_VSPHERE_USERNAME='administrator@vsphere.local'
export EKSA_VSPHERE_PASSWORD='my-password'
eksctl anywhere create cluster -f cluster.yaml

Cluster Lifecycle: Create, Upgrade, and Scale

EKS Anywhere manages the full cluster lifecycle using eksctl anywhere. Cluster state is stored in a management cluster that persists after workload cluster creation.

OperationCommandNotes
Create clustereksctl anywhere create cluster -f cluster.yamlBootstraps temp management cluster, provisions nodes
Upgrade K8s versioneksctl anywhere upgrade cluster -f cluster.yamlRolls control plane then worker nodes
Scale worker nodesEdit cluster.yaml count field, then upgrade clusterDeclarative - change spec, apply with upgrade command
Delete clustereksctl anywhere delete cluster -f cluster.yamlRemoves VMs and cluster resources
Check versioneksctl anywhere versionShows CLI and supported EKS Distro versions
bash
# cluster.yaml - minimal vSphere cluster spec
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
  name: my-cluster
spec:
  kubernetesVersion: "1.29"
  controlPlaneConfiguration:
    count: 3
    machineGroupRef:
      kind: VSphereMachineConfig
      name: control-plane
  workerNodeGroupConfigurations:
    - count: 3
      machineGroupRef:
        kind: VSphereMachineConfig
        name: worker-node
  externalEtcdConfiguration:
    count: 3
    machineGroupRef:
      kind: VSphereMachineConfig
      name: etcd
  networking:
    cniConfig:
      cilium: {}
  clusterNetwork:
    pods:
      cidrBlocks: ["192.168.0.0/16"]
    services:
      cidrBlocks: ["10.96.0.0/12"]
💡

EKS Anywhere uses a cluster-API-based approach internally. A temporary bootstrap cluster is created locally, provisions your target cluster nodes, then the management capability is moved into the cluster itself. The bootstrap cluster is deleted after creation.

EKS Connector: Visibility in the AWS Console

EKS Connector allows you to register any Kubernetes cluster (including EKS Anywhere, on-prem K8s, GKE, AKS) with the AWS console for visibility and management - without giving AWS direct access to your infrastructure.

FeatureWith EKS ConnectorWithout EKS Connector
AWS Console visibilityView workloads, nodes, namespaces in consoleNot visible in AWS console
CloudWatch Container InsightsMetrics and logs from on-prem clusterManual Prometheus/Grafana setup required
IAM access to clusterMap IAM identities to K8s RBAC via consoleManual kubeconfig distribution only
Network requirementOutbound HTTPS from cluster to AWS endpointFully airgapped if needed
Data sent to AWSMetadata only - no actual workload dataNothing
bash
# Register EKS Anywhere cluster with EKS Connector
# 1. Register the cluster in AWS
aws eks register-cluster \
  --name my-eks-anywhere-cluster \
  --connector-config roleArn=arn:aws:iam::123:role/AmazonEKSConnectorAgentRole,provider=EKS_ANYWHERE

# 2. Apply the connector manifest to the cluster
# Download and apply agent manifest generated by the register command
kubectl apply -f eks-connector.yaml
kubectl apply -f eks-connector-binding.yaml
⚠️

EKS Connector requires outbound network connectivity from your on-prem cluster to the EKS API endpoint (eks.us-east-1.amazonaws.com). If your cluster is fully airgapped, the connector agent cannot establish the connection and the cluster will not appear in the AWS console. Plan firewall rules accordingly.

Curated Packages and Add-ons

EKS Anywhere provides a curated package controller that deploys and manages popular open-source tools in your cluster. These are tested and supported by AWS, reducing the operational burden of managing add-ons on-premises.

PackageCategoryNotes
CoreDNSDNSPre-installed in every cluster
cert-managerCertificate managementAutomated certificate issuance
AWS Load Balancer ControllerIngressWorks with on-prem load balancers via annotations
PrometheusMetricsFull Prometheus stack with AlertManager
GrafanaDashboardsPre-built dashboards for K8s metrics
Fluent BitLog forwardingShip logs to CloudWatch or Splunk
HarborContainer registryOn-prem ECR alternative for airgapped clusters
MetallbLoad balancerBare metal load balancer for on-prem services

Curated packages are pulled from a private ECR registry that is part of the EKS Anywhere subscription. For airgapped environments, packages can be mirrored to a local registry.

💡

EKS Anywhere curated packages require the EKS Anywhere Enterprise subscription for on-premises production use. The subscription also includes support SLAs for cluster issues. Without the subscription, you can still use EKS Anywhere but without AWS support for the add-on packages.

Airgapped Deployments and Registry Mirroring

EKS Anywhere supports fully airgapped deployments where the cluster has no internet connectivity. All container images and packages are mirrored to a local registry.

bash
# Configure registry mirror in cluster.yaml for airgapped deployment
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
  name: airgapped-cluster
spec:
  # ... other spec fields
  registryMirrorConfiguration:
    endpoint: harbor.internal.example.com:443
    port: "443"
    caCertContent: |
      -----BEGIN CERTIFICATE-----
      ... your CA cert ...
      -----END CERTIFICATE-----
    authenticate: true
    mirrorNamespaces:
      - public.ecr.aws
Deployment ModeNetwork RequiredImage SourceUse Case
ConnectedFull internetpublic.ecr.awsDev/test, clusters with internet access
Partially airgappedAWS endpoints onlyCombination of ECR and localRestricted corporate networks
Fully airgappedNoneLocal OCI registry (Harbor, Nexus)Classified, air-gapped facilities, Snow
⚠️

For airgapped deployments, you must mirror ALL required images before cluster creation - including the node OS images, Kubernetes component images, and all curated package images. A partial mirror causes cluster creation to fail partway through with cryptic image pull errors. Use the eksctl anywhere download images command to get the full list.

🎯

Interview Focus Points

  • 1What is the difference between EKS Anywhere, EKS on Outposts, and EKS Distro - when would you choose each?
  • 2What business or compliance requirements would lead a customer to choose EKS Anywhere over cloud EKS?
  • 3How does EKS Anywhere manage cluster lifecycle - explain the bootstrap cluster pattern it uses internally?
  • 4What is EKS Connector and what visibility does it provide without requiring direct AWS access to on-premises infrastructure?
  • 5How would you set up an airgapped EKS Anywhere cluster - what are the prerequisites and pitfalls?
  • 6What are EKS Anywhere curated packages and what is the subscription model for them?
  • 7How does EKS Anywhere handle cluster upgrades - what is the rolling upgrade process?
  • 8What networking considerations are unique to EKS Anywhere compared to cloud EKS (CNI options, load balancers, ingress)?