Frequently asked questions

How many GPUs do I need? Start from model memory (weights + optimizer + activations for training; weights + KV cache for inference) and target throughput — the Recommend and Designer tools estimate this.

MIG or time-slicing? MIG for isolation and multi-tenancy on capable parts; time-slicing for dev/low-priority sharing where isolation isn't required.

InfiniBand or RoCE? InfiniBand for the largest training at lowest latency; RoCEv2 to train on an Ethernet fabric with your existing tooling. Both need a non-blocking, lossless design.

NVIDIA or AMD? It depends on the workload and your ROCm/CUDA ecosystem fit — compare them directly in the catalog rather than assuming.