Company
Date Published
Author
Pierre Cariou, Paul Gottschling
Word count
3120
Language
English
Hacker News points
None

Summary

Pierre Cariou and Paul Gottschling discuss the challenges of managing large volumes of logs in systems that consume significant resources, leading to higher costs. To address these challenges, they recommend implementing log management policies with teams, defining four key questions: what is the most appropriate storage solution, how long should logs spend in different kinds of storage, are there logs that you only need at certain times, and are there logs that you need only as aggregates or samples. The authors explain how to prepare organizations to adopt sustainable log management policies by centralizing logs and setting up log management budgets with teams. They also discuss two approaches for implementing log management policies: excluding all logs from hot storage by default and including all logs in hot storage by default. Datadog's Logging without Limits feature helps streamline log management policies and empowers teams to change these policies on the fly, enabling them to dynamically route logs to hot and cold storage based on their usage.