AWS Containers
EKS Anywhere
Run EKS on your own on-premises infrastructure or VMware vSphere
EKS Anywhere is a deployment option for Amazon EKS that lets you run Kubernetes clusters on your own on-premises infrastructure, VMware vSphere, bare metal, Nutanix, Apache CloudStack, or Snow devices - using the same EKS tooling and APIs as cloud EKS. It gives organizations with data sovereignty, latency, or compliance requirements the ability to run Kubernetes with AWS-supported tools and optionally connect to AWS services. EKS Anywhere is distinct from EKS on Outposts - you fully own and manage the infrastructure while AWS provides the tooling.
EKS Anywhere vs EKS vs Outposts - Understanding the Options
AWS offers multiple ways to run Kubernetes. Choosing the right one requires understanding the infrastructure ownership model, connectivity requirements, and what AWS manages.
| Option | Infrastructure Owner | Control Plane Location | AWS Connection Required | Use Case |
|---|---|---|---|---|
| EKS (Cloud) | AWS | AWS (fully managed) | Yes - you are in AWS | Standard cloud workloads |
| EKS on Outposts (Extended) | AWS (Outposts rack) | AWS region (remote worker nodes on Outposts) | Yes - requires AWS region connectivity | Low-latency apps needing AWS-managed control plane |
| EKS on Outposts (Local) | AWS (Outposts rack) | On-premises (control plane on Outposts) | No - fully offline capable | Disconnected or strictly local compliance |
| EKS Anywhere (vSphere/bare metal) | You | On-premises (you manage nodes) | Optional - airgapped supported | Data sovereignty, existing on-prem infra |
| EKS Distro | You | Your infrastructure | No - just the K8s distribution | DIY - just want the EKS K8s build |
EKS Distro is just the Kubernetes distribution that EKS uses - upstream K8s with AWS patches and security fixes. EKS Anywhere is the full tooling layer (cluster lifecycle management, add-ons) built on top of EKS Distro. You get EKS Distro for free; EKS Anywhere has an optional support subscription.
Infrastructure Providers and Hardware Requirements
EKS Anywhere supports multiple infrastructure providers. Each has different setup requirements and feature parity levels.
| Provider | Status | Min Hardware | Notes |
|---|---|---|---|
| VMware vSphere | GA | vSphere 7.0+, 2 vCPU/8GB RAM per node | Most mature - full feature support |
| Bare Metal | GA | x86_64 servers, Tinkerbell for PXE | Uses Tinkerbell for provisioning |
| Nutanix | GA | Nutanix AOS 6.5+ | Full lifecycle management |
| Apache CloudStack | GA | CloudStack 4.14+ | Community-supported provider |
| AWS Snow | GA | Snowball Edge, Snow Family | For disconnected/edge AWS devices |
| Docker (local) | Development only | Docker Desktop on Mac/Linux | Not for production - dev/test only |
Minimum cluster requirements: 1 admin machine (Linux/Mac), 1 control plane node, 1 worker node. For HA production: 3 control plane nodes + 3+ worker nodes.
# Install eksctl-anywhere CLI
brew install aws/tap/eks-anywhere
# Generate cluster configuration
export CLUSTER_NAME=my-eks-anywhere
export VSPHERE_DATACENTER=dc-01
export VSPHERE_DATASTORE=datastore-01
export VSPHERE_FOLDER=/dc-01/vm/eks-anywhere
export VSPHERE_NETWORK="VM Network"
export VSPHERE_SERVER=vcenter.example.com
eksctl anywhere generate clusterconfig $CLUSTER_NAME \
--provider vsphere > cluster.yaml
# Review and edit cluster.yaml, then create
export EKSA_VSPHERE_USERNAME='administrator@vsphere.local'
export EKSA_VSPHERE_PASSWORD='my-password'
eksctl anywhere create cluster -f cluster.yamlCluster Lifecycle: Create, Upgrade, and Scale
EKS Anywhere manages the full cluster lifecycle using eksctl anywhere. Cluster state is stored in a management cluster that persists after workload cluster creation.
| Operation | Command | Notes |
|---|---|---|
| Create cluster | eksctl anywhere create cluster -f cluster.yaml | Bootstraps temp management cluster, provisions nodes |
| Upgrade K8s version | eksctl anywhere upgrade cluster -f cluster.yaml | Rolls control plane then worker nodes |
| Scale worker nodes | Edit cluster.yaml count field, then upgrade cluster | Declarative - change spec, apply with upgrade command |
| Delete cluster | eksctl anywhere delete cluster -f cluster.yaml | Removes VMs and cluster resources |
| Check version | eksctl anywhere version | Shows CLI and supported EKS Distro versions |
# cluster.yaml - minimal vSphere cluster spec
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: my-cluster
spec:
kubernetesVersion: "1.29"
controlPlaneConfiguration:
count: 3
machineGroupRef:
kind: VSphereMachineConfig
name: control-plane
workerNodeGroupConfigurations:
- count: 3
machineGroupRef:
kind: VSphereMachineConfig
name: worker-node
externalEtcdConfiguration:
count: 3
machineGroupRef:
kind: VSphereMachineConfig
name: etcd
networking:
cniConfig:
cilium: {}
clusterNetwork:
pods:
cidrBlocks: ["192.168.0.0/16"]
services:
cidrBlocks: ["10.96.0.0/12"]EKS Anywhere uses a cluster-API-based approach internally. A temporary bootstrap cluster is created locally, provisions your target cluster nodes, then the management capability is moved into the cluster itself. The bootstrap cluster is deleted after creation.
EKS Connector: Visibility in the AWS Console
EKS Connector allows you to register any Kubernetes cluster (including EKS Anywhere, on-prem K8s, GKE, AKS) with the AWS console for visibility and management - without giving AWS direct access to your infrastructure.
| Feature | With EKS Connector | Without EKS Connector |
|---|---|---|
| AWS Console visibility | View workloads, nodes, namespaces in console | Not visible in AWS console |
| CloudWatch Container Insights | Metrics and logs from on-prem cluster | Manual Prometheus/Grafana setup required |
| IAM access to cluster | Map IAM identities to K8s RBAC via console | Manual kubeconfig distribution only |
| Network requirement | Outbound HTTPS from cluster to AWS endpoint | Fully airgapped if needed |
| Data sent to AWS | Metadata only - no actual workload data | Nothing |
# Register EKS Anywhere cluster with EKS Connector
# 1. Register the cluster in AWS
aws eks register-cluster \
--name my-eks-anywhere-cluster \
--connector-config roleArn=arn:aws:iam::123:role/AmazonEKSConnectorAgentRole,provider=EKS_ANYWHERE
# 2. Apply the connector manifest to the cluster
# Download and apply agent manifest generated by the register command
kubectl apply -f eks-connector.yaml
kubectl apply -f eks-connector-binding.yamlEKS Connector requires outbound network connectivity from your on-prem cluster to the EKS API endpoint (eks.us-east-1.amazonaws.com). If your cluster is fully airgapped, the connector agent cannot establish the connection and the cluster will not appear in the AWS console. Plan firewall rules accordingly.
Curated Packages and Add-ons
EKS Anywhere provides a curated package controller that deploys and manages popular open-source tools in your cluster. These are tested and supported by AWS, reducing the operational burden of managing add-ons on-premises.
| Package | Category | Notes |
|---|---|---|
| CoreDNS | DNS | Pre-installed in every cluster |
| cert-manager | Certificate management | Automated certificate issuance |
| AWS Load Balancer Controller | Ingress | Works with on-prem load balancers via annotations |
| Prometheus | Metrics | Full Prometheus stack with AlertManager |
| Grafana | Dashboards | Pre-built dashboards for K8s metrics |
| Fluent Bit | Log forwarding | Ship logs to CloudWatch or Splunk |
| Harbor | Container registry | On-prem ECR alternative for airgapped clusters |
| Metallb | Load balancer | Bare metal load balancer for on-prem services |
Curated packages are pulled from a private ECR registry that is part of the EKS Anywhere subscription. For airgapped environments, packages can be mirrored to a local registry.
EKS Anywhere curated packages require the EKS Anywhere Enterprise subscription for on-premises production use. The subscription also includes support SLAs for cluster issues. Without the subscription, you can still use EKS Anywhere but without AWS support for the add-on packages.
Airgapped Deployments and Registry Mirroring
EKS Anywhere supports fully airgapped deployments where the cluster has no internet connectivity. All container images and packages are mirrored to a local registry.
# Configure registry mirror in cluster.yaml for airgapped deployment
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: airgapped-cluster
spec:
# ... other spec fields
registryMirrorConfiguration:
endpoint: harbor.internal.example.com:443
port: "443"
caCertContent: |
-----BEGIN CERTIFICATE-----
... your CA cert ...
-----END CERTIFICATE-----
authenticate: true
mirrorNamespaces:
- public.ecr.aws| Deployment Mode | Network Required | Image Source | Use Case |
|---|---|---|---|
| Connected | Full internet | public.ecr.aws | Dev/test, clusters with internet access |
| Partially airgapped | AWS endpoints only | Combination of ECR and local | Restricted corporate networks |
| Fully airgapped | None | Local OCI registry (Harbor, Nexus) | Classified, air-gapped facilities, Snow |
For airgapped deployments, you must mirror ALL required images before cluster creation - including the node OS images, Kubernetes component images, and all curated package images. A partial mirror causes cluster creation to fail partway through with cryptic image pull errors. Use the eksctl anywhere download images command to get the full list.
Interview Focus Points
- 1What is the difference between EKS Anywhere, EKS on Outposts, and EKS Distro - when would you choose each?
- 2What business or compliance requirements would lead a customer to choose EKS Anywhere over cloud EKS?
- 3How does EKS Anywhere manage cluster lifecycle - explain the bootstrap cluster pattern it uses internally?
- 4What is EKS Connector and what visibility does it provide without requiring direct AWS access to on-premises infrastructure?
- 5How would you set up an airgapped EKS Anywhere cluster - what are the prerequisites and pitfalls?
- 6What are EKS Anywhere curated packages and what is the subscription model for them?
- 7How does EKS Anywhere handle cluster upgrades - what is the rolling upgrade process?
- 8What networking considerations are unique to EKS Anywhere compared to cloud EKS (CNI options, load balancers, ingress)?