Scaling the GitLab database

GitLab initially relied on a single PostgreSQL database server and a replica for disaster recovery, but as demand grew, this setup faced limitations, prompting GitLab to explore alternative solutions for both GitLab.com and self-managed instances. To manage database load and enhance resilience, GitLab adopted several strategies, including connection pooling with PgBouncer and load balancing using custom-built solutions, rather than sharding, due to the low write load and the complexity and resource demands of sharding. PgBouncer was chosen over Pgpool due to its simplicity and effectiveness in connection pooling, allowing GitLab to reduce active PostgreSQL connections significantly, from hundreds to around 10-20, through transaction pooling. For load balancing, GitLab developed a custom solution to direct queries efficiently across primary and hot-standby servers, ensuring seamless read and write operations while reducing CPU utilization significantly. GitLab also collaborated with Crunchy Data to optimize PostgreSQL performance, achieving lower CPU and memory usage. The integration of these strategies allowed GitLab to scale effectively and maintain high availability, with plans to further enhance the system but without intentions to make their load balancing solution available as a standalone product.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.