Company
Date Published
Author
David Cramer
Word count
1751
Language
English
Hacker News points
None

Summary

We use PostgreSQL at Sentry as our core database, with two clusters operating cold-standby replicas. Most data is stored in SQL, except for large event blobs which are written to Riak. We see a pattern of heavily-duplicated errors and unique errors creating new aggregates. To alleviate locking contention, we take a buffered writes approach, aggregating writes over time and flushing them after an interval of 10 seconds. This allows us to greatly increase our throughput by sacrificing strong consistency. We use Redis as a buffer to store the pending writes, with a simple schema that stores entity attributes and keys in two structures: a hash per entity and a set of hashes which need flushed. The Redis implementation ensures that only a single row in SQL is being updated at once, alleviating most locking contention.