highDetection & drivers

Driver install failure

The GPU driver (or operator driver pod) fails to build or load.

Scope
Host/operator level. Often a kernel-headers or Secure Boot problem.
Check recent changes
  • ·Kernel upgrade
  • ·New OS image
  • ·Secure Boot enabled
Diagnostics

Read the driver build log

bash
kubectl -n gpu-operator logs -l app=nvidia-driver-daemonset --tail=100

Expect: A DKMS/build error pinpoints missing headers or a signing issue.

Kernel vs. headers

bash
uname -r; rpm -q kernel-headers 2>/dev/null || dpkg -l | grep linux-headers

Expect: Headers match the running kernel.

Likely causes & fixes
  • high likelihoodKernel headers missing or mismatched

    Fix · Install headers for the exact running kernel; pin kernel versions.

  • medium likelihoodSecure Boot rejects the unsigned module

    Fix · Enroll a signing key (MOK) or disable Secure Boot per policy.

  • low likelihoodUnsupported kernel/driver combo

    Fix · Move to a validated driver+kernel baseline.

Validation
  • ✓Driver loads; `nvidia-smi` works on the host
Escalation

Persistent build failures on a supported baseline → vendor support.

Data to collect first
  • ·Driver pod / installer logs
  • ·Running kernel vs. installed headers
  • ·Secure Boot state