Cloud-managedManagedNVIDIA

Amazon EKS

Amazon Web Services

Managed Kubernetes on AWS. GPU nodes come from GPU instance types with either the EKS GPU-optimized AMI (drivers baked in) or the NVIDIA GPU Operator managing drivers.

Bootstrap with Amazon EKS
Enablement

NVIDIA GPU Operator or EKS GPU AMI + device plugin

Driver install

EKS GPU-optimized AMI ships drivers; or let the GPU Operator install them on a standard AMI.

Container runtime

containerd (NVIDIA runtime configured by the operator or AMI).

Scheduling

Default scheduler with nvidia.com/gpu resource; node groups per GPU type; Karpenter for GPU-aware autoscaling.

Partitioning

MIG on A100/H100; time-slicing via the operator's device-plugin config.

Networking

VPC CNI; EFA for high-throughput RDMA on supported instances (add the EFA device plugin).

Upgrades

Managed node group / Karpenter rollouts; coordinate AMI driver version with CUDA needs.

GPU enablement
  1. 1

    Create a GPU managed node group

    Use a GPU instance type (e.g. g5/p4d/p5) with the GPU-optimized AMI, or a standard AMI if the operator will install drivers.

  2. 2

    Install the GPU Operator (standard AMI path)

    Skip if using the GPU AMI, which already includes drivers and the device plugin.

    bash
    helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm install --wait gpu-operator nvidia/gpu-operator \
  -n gpu-operator --create-namespace
  3. 3

    Label / taint for isolation

    Taint GPU nodes so only GPU workloads land there.

    bash
    kubectl taint nodes -l node.kubernetes.io/instance-type=g5.xlarge \
  nvidia.com/gpu=present:NoSchedule
Validate the enablement
verify GPUs
kubectl get nodes -o custom-columns=NAME:.metadata.name,GPU:.status.allocatable.'nvidia\.com/gpu'
kubectl -n gpu-operator get pods
kubectl run cuda-check --rm -it --restart=Never \
  --image=nvidia/cuda:12.4.1-base-ubuntu22.04 \
  --limits=nvidia.com/gpu=1 -- nvidia-smi
Known limitations
  • ·Driver version is coupled to the AMI unless the operator manages it.
  • ·EFA/RDMA is instance-type specific and needs its own device plugin.
Best for

Teams standardized on AWS wanting managed control-plane with flexible GPU instance families.