Giving Inngest's Queue a Bigger Brain (and a Backup Generator): Our Migration to FoundationDB, Part 1

Inngest is migrating its internal job queue from Valkey, an in-memory Redis-compatible store, to FoundationDB to address limits in memory capacity, durability, scalability, and tenant fairness. The queue schedules every function run across billing tiers and must maintain low latency, high throughput, ordering, and isolation among customers, but Valkey’s memory-bound, single-threaded architecture risked cluster-wide outages when capacity was exhausted and could potentially lose unpersisted queue state during failures. After extending Valkey’s lifespan through manual shard scaling, a separate constraint service, and attempts at per-key fairness, Inngest concluded that its remaining limitations were inherent to the storage engine rather than solvable with additional queue-level changes. FoundationDB was selected for its disk-backed ACID transactions, horizontal scaling, multi-threaded capacity, ordered key-value model, and separation of transaction and storage paths, while its use by systems such as Apple CloudKit and Snowflake provided additional evidence of its maturity. The new architecture will use separate FoundationDB clusters for free, self-serve, and enterprise tiers, with automatically assigned logical shards and optional dedicated enterprise isolation, and deployment will proceed gradually from free accounts to enterprise customers with rollback capability and extensive benchmarking.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.