Scaling Grafana Mimir to 500 million active series on customer infrastructure with Grafana Enterprise Metrics

In an effort to address the growing demand for scalable data storage solutions, Grafana Labs conducted extensive load testing on Grafana Mimir, an open-source time series database, and Grafana Enterprise Metrics (GEM) to demonstrate their ability to handle up to 500 million active series on customer infrastructure. The initiative involved partnering with a customer who required a sustained load test at this scale, which had not been previously achieved by other vendors. By using an AWS environment that mirrored the customer's setup, Grafana Labs incrementally scaled the load, starting from 50 million active series and advancing to 500 million, while addressing challenges such as authentication bugs and AWS quota limits. The testing process involved optimizing query sharding, adjusting ingester configurations, and enhancing performance efficiencies, which resulted in significant infrastructure cost savings. Throughout the testing phases, Grafana Labs showcased the capability of GEM to manage high cardinality queries and rapid data ingestion rates, ultimately proving GEM's scalability and reliability for large-scale metric management in on-premises environments.