How we built a real-time service that handles 20,000 updates per second

Building production-grade AI agents necessitates more than merely utilizing OpenAI's API, requiring long-running tasks, durability, queues, and crucially, real-time user feedback. Trigger.dev's Realtime service addresses these needs by processing over 20,000 run updates per second and ingesting over 500GB of data daily from Postgres, achieved through technical decisions like using Postgres replication slots over WebSockets. The system allows developers to make background tasks visible, providing users with real-time updates, such as progress bars, to ensure transparency and reliability. Authentication is optimized with JWTs to avoid frequent database queries, while Redis Lua scripts manage rate limiting. The architecture prioritizes scalability and performance, maintaining low latency and strong consistency guarantees, with ElectricSQL effectively managing data distribution while leveraging Postgres as the source of truth for real-time updates.