Splitting production databases with minimal downtime

Sentry has implemented strategies to scale its Postgres relational datastore to handle high write throughput by leveraging both vertical and horizontal scaling. The company operates Postgres using typical replication topologies, with applications connecting through a pgbouncer sidecar and each Postgres primary having multiple replicas. To scale horizontally, Sentry employs product area sharding, which allows infrastructure segmentation along product lines and isolation of failure domains, preventing issues in one product from affecting others. This method has enabled Sentry to manage its largest US region with 14 Postgres primaries and facilitated multi-region deployments, such as the EU region, by partitioning Postgres primaries along silo boundaries. The application, built with Django, uses database routers extensively, allowing for effective management of multiple Postgres clusters. Preparing for database splits involves analyzing transactional boundaries and selectively removing foreign key constraints, as seen in the movement of the User model to a control silo. The process includes updating database mappings, reconfiguring connections, and eventually performing physical separation by creating and promoting new Postgres primaries. This approach to sharding by domain has effectively addressed write capacity challenges, with future plans potentially exploring customer-level sharding to further optimize transactional boundaries.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.