Introducing the Baseten Delivery Network: Fast cold starts for big models

The Baseten Delivery Network (BDN) has been launched to significantly reduce cold start times for large-scale models, achieving 2-3x faster initialization through multi-tier caching and single-flight downloads, which address issues such as thundering herd problems during burst scaling. BDN is integrated with the Baseten Inference Stack and is designed to overcome common challenges associated with cold starts, including hardware provisioning and weight downloading—particularly for models with tens to hundreds of billions of parameters. By mirroring weights to infrastructure at push time, employing a multi-tier cache system, and ensuring single-flight weight downloads, BDN eliminates dependencies on third-party services and reduces bandwidth contention, ensuring consistent and rapid scaling even in high-demand scenarios. Additionally, BDN accelerates the entire inference loop by distributing various deployment artifacts, providing immediate improvements in cold start times and efficiency gains throughout the model deployment lifecycle, and is now available to all Baseten Cloud customers.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.