Cold starts just got hot

Neon's engineering team has significantly reduced the cold start time for compute resources, cutting it down from an average of 3-6 seconds to around 500 milliseconds. This improvement was achieved by re-evaluating the problem and applying two main optimization strategies: reconfiguring only when necessary and utilizing compute pools. The first strategy involves avoiding unnecessary configuration updates unless a compute is idle and requires a health check, while the second strategy focuses on having pre-started computes ready to be configured on demand, thus bypassing the longer start times associated with on-demand setups. Additionally, improvements in networking, caching, and concurrency, along with a new configuration system, have further decreased start times. The ongoing efforts include autoscaling and optimizing resource allocation to maintain efficiency, with the ultimate goal of achieving consistent speeds across all regions. These enhancements are part of a broader initiative to improve Neon’s serverless offerings by making cold starts a thing of the past and ensuring faster and more efficient compute resource management.