Home / Companies / Grafana Labs / Blog / Post Details
Content Deep Dive

Avoid dropped logs due to out-of-order timestamps with a new Loki feature

Blog post from Grafana Labs

Post Details
Company
Date Published
Author
Owen Diehl
Word Count
562
Language
English
Hacker News Points
-
Summary

In the upcoming v2.4 release of Loki, the requirement for log entries to arrive in order by timestamp will be removed, addressing one of the most-requested features. This change allows out-of-order writes to be accepted, simplifying setups where multiple instances of an app or complex systems result in logs arriving slightly out of order, which previously led to data being dropped. The new feature can be configured easily, with a negligible performance cost, and applies either to an entire Loki cluster or on a tenant-by-tenant basis. Users of Grafana Cloud, a managed platform that integrates metrics, logs, traces, and dashboards, can already benefit from this functionality, as out-of-order writes up to one hour old are accepted. This advancement aims to accommodate various logging scenarios, particularly in high-load systems or those using ephemeral jobs like Lambdas, where maintaining strict log ordering is challenging.