GPU Job Queueing with Kueue and DRA: Scheduling AI Workloads Without Idle Capacity

Kueue is a Kubernetes-native admission controller that queues GPU batch workloads until sufficient quota and capacity are available, aiming to prevent indefinitely pending pods, improve visibility, enforce multi-team fairness, and support coordinated admission for distributed jobs. It uses ResourceFlavors to represent hardware pools, ClusterQueues to define guaranteed quotas and borrowing or lending limits, LocalQueues as namespace-level submission points, and Workloads as the units evaluated for admission; cohorts and preemption policies allow unused GPU capacity to be shared while preserving each team’s allocation. Dynamic Resource Allocation, generally available in Kubernetes 1.34, replaces simple GPU-count requests with attribute-based ResourceClaims that can specify requirements such as memory, architecture, compute capability, or MIG partition profiles, allowing more precise device matching and potentially reducing wasted capacity. The text recommends MIG for isolated single-GPU workloads, full GPUs for NVLink-dependent distributed training, and MPS for some shared inference scenarios, while noting that Kueue and DRA require integration and may need retry mechanisms when device availability changes after admission. It contrasts Kueue’s Kubernetes-native approach with Volcano’s separate scheduler and stronger pod-placement gang semantics, and cites Cast AI fleet data and customer-oriented estimates suggesting low average GPU utilization and potential savings from combining queuing, device partitioning, and attribute-aware autoscaling.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.