NVIDIA Vera CPUs double throughput for Modal Sandboxes

Modal is evaluating NVIDIA’s Arm-compatible Vera CPU as a potential platform for its Sandbox service, a shift that would require substantial changes from its historically x86-only infrastructure and image-building systems. The company argues that sandbox workloads are especially difficult to schedule because hundreds of heterogeneous, bursty containers and virtual machines share hosts, creating contention and noisy-neighbor effects despite Modal’s use of burstable capacity rather than fixed peak reservations. To assess performance, Modal developed Agentless Agent Bench, a suite of 50 network-free Terminal-Bench tasks representing agent and reinforcement-learning activities such as compilation, file search, Git operations, and device emulation. While Vera’s single-task latency was only modestly better or sometimes comparable with competing CPUs, its throughput reportedly scaled much more effectively under heavy concurrent load, reaching up to twice that of a comparable Intel Granite Rapids system in early tests. Modal attributes this result largely to Vera’s memory bandwidth, which it says enables denser container packing without the throughput plateaus and latency degradation observed on other platforms, potentially improving agent responsiveness, reducing CPU and GPU idle costs, and increasing available Sandbox scale.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.