November 2026 Summaries
7 posts from Cast AI
Filter
Month:
Year:
Post Summaries
Back to Blog
No summary generated yet.
Nov 16, 2026
3,485 words in the original blog post.
No summary generated yet.
Nov 16, 2026
2,790 words in the original blog post.
No summary generated yet.
Nov 16, 2026
2,734 words in the original blog post.
No summary generated yet.
Nov 16, 2026
2,804 words in the original blog post.
GPU utilization in Kubernetes is often far below potential, with Cast AI’s 2026 report citing an average sustained utilization of 5% across analyzed production clusters, driven mainly by oversized GPU requests, exclusive one-pod-per-GPU allocation, fragmented scheduling, static node pools, and unqueued batch work. Effective measurement requires per-pod DCGM metrics that distinguish streaming multiprocessor activity, VRAM use, and memory-copy activity, since the commonly viewed nvidia-smi utilization figure can register intermittent kernel activity while a device performs little useful compute. Recommended improvements are to right-size CPU, memory, and GPU-related resource requests using measured p95 usage; share GPUs through MIG, MPS, or time-slicing depending on isolation and latency needs; use topology-aware bin-packing; autoscale nodes after capacity has been consolidated; and queue training or batch jobs through tools such as Kueue and gang scheduling. MIG provides hardware-isolated partitions, MPS enables flexible concurrent CUDA contexts with softer isolation, and time-slicing increases density but offers no memory or compute isolation, making it less suitable for latency-sensitive workloads. Autoscaling tools such as Karpenter and workload-level scaling can reduce off-peak capacity, but accurate requests and consolidated placement are necessary for scale-down to work effectively. The text emphasizes continuous automation of measurement, rightsizing, packing, scaling, and cost attribution, arguing that manual tuning becomes outdated as workloads and demand patterns change.
Nov 11, 2026
2,959 words in the original blog post.
Idle Kubernetes capacity is the gap between resource requests and actual workload consumption, which can leave clusters apparently full to the scheduler despite low real utilization because Kubernetes schedules against declared requests rather than usage. Drawing on Cast AI’s 2026 report covering tens of thousands of clusters, the material cites average CPU utilization of 8%, memory utilization of 20%, CPU overprovisioning of 69%, and memory overprovisioning of 79%, attributing much of this waste to configuration choices. It identifies recurring sources of idle spend as oversized workload requests, inactive pods or replicas, underpacked nodes, orphaned persistent volumes and load balancers, and forgotten namespaces. Recommended analysis compares each container’s CPU and memory requests with P95 usage over a representative period, ranks inefficient workloads, and validates historical peaks, traffic patterns, ownership, and storage retention policies before changes. The proposed remediation sequence is to rightsize requests with appropriate headroom, consolidate accurately sized workloads onto fewer nodes while respecting availability constraints, and only then remove abandoned resources after verification and backups. Continuous monitoring, chargeback visibility, autoscaling, bin-packing, and automated rightsizing are presented as ways to prevent idle capacity from returning as workloads, charts, and operational requirements evolve.
Nov 11, 2026
3,437 words in the original blog post.
CastPilot MCP is a generally available, read-only integration that lets MCP-compatible AI clients retrieve Cast AI cluster, policy, cost, and database-optimization data directly, enabling engineers to investigate production questions and prepare code or configuration changes without manually collecting context. Examples include reviewing cluster policies during on-call work, combining MCP data with custom simulations for Evictor dry runs, applying Database Optimizer Performance Advisor recommendations to repository code, and tracing cost increases through workloads and allocation groups. Its architecture generates tools and authorization rules from Cast AI’s OpenAPI specification and existing RBAC model, then applies a curated, read-focused filter to limit exposure and reduce tool-selection complexity. Agent-facing overlays provide clearer names, descriptions, identifier resolution, response shaping, and guided investigation prompts, while agentgateway handles authentication, permission-based tool filtering, and API forwarding. The system emphasizes user consent for ambiguous or cross-organization access, avoids recording tool-response contents in telemetry, and is designed to help users bring Cast AI insights into their preferred development workflows while retaining existing approval and validation processes.
Nov 10, 2026
3,726 words in the original blog post.