Company
Date Published
Author
MongoDB
Word count
1556
Language
English
Hacker News points
None

Summary

Processing 2 Billion Documents A Day And 30TB A Month With MongoDB` The company Server Density processes over 30TB/month of data points from its monitoring services, which are fed into MongoDB in real-time for analysis. To scale their MongoDB database, they use dedicated hardware and SSDs to eliminate CPU steal and shared storage issues, allowing for improved concurrency by splitting data across multiple databases on separate SSDs. They also employ hash-based sharding to ensure uniform distribution of shard chunks. Additionally, they utilize TTL indexes to automatically delete old data, which helps with disk usage efficiency and reduces batch processing needs. Careful query and schema design is also crucial to avoid performance issues such as document size increases and network throughput limitations. By implementing these strategies, Server Density achieves thousands of write operations per second and response times within 10ms while using a write concern of 1.