mediumPerformance

Low GPU utilization

GPUs sit underutilized while work is queued or running — usually the pipeline, not the GPU.

Scope
Performance/efficiency. The GPU is rarely the bottleneck when utilization is low.
Check recent changes
  • ·New data pipeline
  • ·Batch size change
  • ·Different storage backend
Diagnostics

Compute vs. data movement

text
# In Grafana/DCGM compare:
#   DCGM_FI_DEV_GPU_UTIL   (compute)
#   DCGM_FI_DEV_MEM_COPY_UTIL (data movement)

Expect: High copy + low compute ⇒ input-bound.

Likely causes & fixes
  • high likelihoodData loading / preprocessing bottleneck

    Fix · Add workers, prefetch, cache; move data closer.

  • high likelihoodBatch size too small

    Fix · Increase batch / use gradient accumulation.

  • medium likelihoodCPU or network bound

    Fix · Scale CPU/network or overlap comms.

  • low likelihoodPoor packing

    Fix · Consolidate small jobs via MIG/time-slicing.

Validation
  • ✓GPU utilization rises toward the 60–95% band
Escalation

None — a tuning exercise.

Data to collect first
  • ·GPU util vs. memory-copy util
  • ·Data-loader/CPU metrics
  • ·Storage read throughput
Related alerts
low-gpu-util