Company
Date Published
Author
Jon Kartago Lamida
Word count
1576
Language
English
Hacker News points
None

Summary

Grafana Mimir has introduced the Mimir Query Engine (MQE) as a solution to address the high memory consumption issues experienced with the Prometheus PromQL engine in its time series database. As of Grafana Mimir 2.17, MQE is the default query engine and is designed to offer faster and more memory-efficient performance by processing queries in a streaming manner, only loading data into memory as needed, rather than all at once. This approach significantly reduces memory fluctuations and improves query execution speeds, while maintaining 100% compatibility with PromQL features, allowing all existing queries and dashboards to function seamlessly. MQE's enhancements include memory-efficient techniques like time splitting, sharding, and chunks streaming, which optimize memory use across queries. Testing for correctness has been thorough, ensuring MQE's results match those from Prometheus' engine through unit tests and staged parallel query paths. The introduction of MQE marks an important step in improving the scalability and reliability of Grafana Mimir, with a focus on better handling large-scale data queries while maintaining accurate and consistent results.