Company
Date Published
Author
Jesús Vázquez and Ganesh Vernekar
Word count
1281
Language
English
Hacker News points
None

Summary

Grafana Mimir 2.2 introduces experimental support for out-of-order sample ingestion, addressing the traditional limitation of Prometheus TSDB, which only accepts in-order samples. This feature caters to use cases like IoT devices, message buses, and isolated Prometheus instances that require the ingestion of older metrics. The design involves storing the last 30 out-of-order samples in memory, utilizing a Write-Behind-Log (WBL) to record ingested samples, and compacting these samples into persistent blocks. Out-of-order samples are managed using a new configuration parameter, out_of_order_time_window, allowing users to specify the permissible age of these samples. This capability is being tested in Grafana Cloud and will be enabled for all users after further testing. The feature is developed in the grafana/mimir-prometheus GitHub repository, allowing for experimentation and problem-solving before contributing upstream. While the feature shows varied performance impacts, including increased CPU usage during high rates of out-of-order sample processing, memory usage remains stable. This advancement positions Grafana Mimir as a versatile time series database capable of integrating with various metrics systems like Graphite, Datadog, Influx, and Prometheus.