How Snorkel runs realistic agent simulations with Modal

Snorkel, an AI data lab that develops datasets and benchmarks for training and evaluating AI systems, uses Modal’s infrastructure to run inference and millions of agent simulations each month. To assess agents realistically across coding, infrastructure, and machine-learning tasks, Snorkel relies on Modal’s CPU, GPU, and VM sandboxes to recreate production-like environments, including multi-service systems and GPU-enabled workloads, while supporting peaks of tens of thousands of sandboxes per hour. Modal’s resource-bursting capability allows sandboxes to expand CPU or memory use when needed, reducing failures caused by under-provisioning and helping Snorkel lower its overall sandbox costs by about 20 percent. Snorkel also uses Modal’s shared and dedicated inference endpoints to rapidly evaluate open-weight and custom fine-tuned models without requiring extensive infrastructure involvement. Having adopted several Modal capabilities over roughly six months, Snorkel plans to consider the platform for training as well, potentially supporting a complete workflow from model training through evaluation.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.