The Constraint API: Scaling flow control beyond millions of RPS

Inngest has introduced the Constraint API, a new system designed to enhance flow control features such as concurrency, throttling, and rate limiting, by decoupling constraint enforcement from the queue. Previously, flow control was tightly integrated with the queue, leading to challenges in scaling and flexibility, particularly with the introduction of Durable Endpoints. The Constraint API provides a dedicated service to handle capacity management through a system of capacity leases, which automatically expire to reclaim resources if necessary, enabling idempotent and atomic operations without manual intervention. This decoupling simplifies the queue's role, optimizes performance, and allows constraint enforcement to scale independently, providing lower latency, better visibility, and enabling flow control for Durable Endpoints. Looking forward, Inngest plans to migrate its queue infrastructure to FoundationDB, which will support greater scalability and consistency, further enhancing throughput and reliability. This strategic shift, facilitated by the Constraint API, sets the stage for substantial improvements in Inngest's infrastructure capabilities.