HPC & scientific computing (Slurm + MPI)
Tightly-coupled simulation — CFD, molecular dynamics, weather, genomics — run as MPI jobs, often under Slurm alongside or bridged to Kubernetes.
Architecture
MPI ranks run across GPU nodes over a low-latency RDMA fabric, coordinated by Slurm (or a K8s batch scheduler). FP64 matters for many HPC codes, so parts with strong double-precision (MI250X/MI300A, H100) shine. Slurm and Kubernetes coexist via Slinky / the Slurm operator, or Slurm bursts into K8s node pools.
Sizing
Scale by problem size and the parallel efficiency of the code; FP64 throughput and interconnect bandwidth usually dominate over FP16.
Networking
InfiniBand (often with SHARP) or lossless RoCEv2; GPUDirect RDMA; MPI tuned for the topology.
Storage
Parallel file system (Lustre / GPFS / BeeGFS) for scratch and results; high aggregate throughput.
Scaling
Gang/backfill scheduling; Slurm partitions map to GPU node pools. Bridge to K8s with Slinky or run Slurm-in-Kubernetes for a unified pool.
Monitoring
Job efficiency, MPI/collective timings, FP64 utilization, fabric counters, queue wait and backfill.
#!/bin/bash
#SBATCH --job-name=cfd-sim
#SBATCH --partition=gpu
#SBATCH --nodes=4
#SBATCH --gpus-per-node=8
#SBATCH --ntasks-per-node=8
#SBATCH --time=04:00:00
srun --mpi=pmix ./solver --input case.dat
# On Kubernetes, Slinky / the Slurm operator maps this partition to a GPU node pool.