Cloud-managedManagedNVIDIAAMD

Azure AKS

Microsoft Azure

Managed Kubernetes on Azure. GPU node pools can use the AKS GPU image or the NVIDIA GPU Operator; AMD Instinct is available on select VM sizes.

Bootstrap with Azure AKS
Enablement

NVIDIA GPU Operator or AKS GPU node pool image

Driver install

AKS can skip its built-in driver install so the GPU Operator manages drivers, or use the GPU device-plugin image.

Container runtime

containerd with the NVIDIA runtime.

Scheduling

Default scheduler; GPU node pools with labels/taints; cluster autoscaler per pool.

Partitioning

MIG and time-slicing via the operator; check VM size support.

Networking

Azure CNI; InfiniBand on ND-series VMs for RDMA training fabrics.

Upgrades

Node-pool image upgrades; align GPU driver with workload CUDA/ROCm versions.

GPU enablement
  1. 1

    Add a GPU node pool

    Create a node pool on an NC/ND VM size; use --skip-gpu-driver-install if the operator will manage drivers.

    bash
    az aks nodepool add -g $RG --cluster-name $AKS \
  -n gpupool --node-vm-size Standard_NC24ads_A100_v4 \
  --node-count 2 --skip-gpu-driver-install
  2. 2

    Install the GPU Operator

    Manages drivers, container runtime, and the device plugin.

    bash
    helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update
helm install --wait gpu-operator nvidia/gpu-operator \
  -n gpu-operator --create-namespace
  3. 3

    AMD Instinct path

    On MI300X VM sizes, install the AMD GPU Operator instead of the NVIDIA one.

Validate the enablement
verify GPUs
kubectl get nodes -o custom-columns=NAME:.metadata.name,GPU:.status.allocatable.'nvidia\.com/gpu'
kubectl -n gpu-operator get pods
kubectl run cuda-check --rm -it --restart=Never \
  --image=nvidia/cuda:12.4.1-base-ubuntu22.04 \
  --limits=nvidia.com/gpu=1 -- nvidia-smi
Known limitations
  • ·GPU VM sizes are region- and quota-limited.
  • ·InfiniBand requires the right VM size and image.
Best for

Azure-native platforms, including mixed NVIDIA/AMD via VM size selection.