Benchmarking How Workflow Execution Scales on Postgres

The blog post explores the scalability of a single Postgres server for executing durable workflows, focusing on its write performance, which is crucial for workflow execution. Through benchmarking, it is demonstrated that a single Postgres server can sustain a throughput of 144,000 writes per second, equivalent to processing 43,000 workflows per second, translating to 12 billion writes or 4 billion workflows per day, which is sufficient for most use cases. Experiments reveal that the main performance bottleneck is in the write-ahead log (WAL) flushing process, a common issue for write-intensive workloads. In scenarios involving Postgres-backed queues, the bottleneck shifts to lock contention in the workflow_status table, though distributing work across multiple queues can alleviate this to some extent. The findings suggest that Postgres scales well for durable workflows, and additional performance can be achieved by sharding workloads across multiple servers.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.