Company
Date Published
Author
Ed Welch
Word count
2308
Language
English
Hacker News points
None

Summary

Grafana Loki, a logging database, uses labels as both the keyspace for sharding incoming data and the index for log retrieval at query time, making their correct usage crucial for successful application. Labels are key-value pairs assigned to logs, and while they seem simple, they can cause issues if misused, particularly with high cardinality, which Loki is not designed to handle due to its preference for long-lived streams and low cardinality. Recent updates include automatic stream sharding for high-volume streams and structured metadata for storing high cardinality data without indexing it, while Bloom filters are being developed to optimize "needle in a haystack" queries. The guide emphasizes using few labels with long-lived and intuitive values, avoiding ephemeral values like trace IDs, to maintain efficient log querying and indexing, and highlights improvements made in Loki's ability to handle large data volumes and reduce operational challenges.