How we scale PgBouncer in ClickHouse Managed Postgres

PgBouncer is a single-threaded connection pooler for Postgres that traditionally uses only one CPU core, limiting throughput on multi-core systems. To maximize resource utilization, ClickHouse Managed Postgres employs a fleet of PgBouncer processes proportional to the number of available CPU cores, with each process using the so_reuseport feature to enable kernel-based load balancing of incoming connections across processes. This setup allows clients to connect to a single endpoint without being aware of the multiple PgBouncer processes in use. A challenge with this approach is query cancellation, as a cancel request can be misdirected to the wrong process, but this is mitigated by a peering mechanism that forwards the request to the correct process. In performance testing on AWS EC2 instances, a single PgBouncer process peaked at about 87k transactions per second, utilizing only one core, while a fleet of 16 processes achieved roughly 336k transactions per second by effectively distributing the workload across available cores. The fleet configuration significantly increased CPU utilization and throughput, demonstrating the efficiency of using multiple PgBouncer processes to avoid bottlenecks in high-concurrency environments. ClickHouse Managed Postgres utilizes this configuration by default, optimizing resource use and enhancing performance for scalable applications.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.