June 2022 Summaries

Observability is fundamentally a data problem, with organizations needing the right data at the right time to effectively troubleshoot issues, yet the challenge lies in predicting which data will be needed. As organizations grow, so does their observability data, raising questions about which data to collect, how to manage it, and how long to retain it. While cost is a major inhibitor of long-term data retention, discarding data solely due to cost concerns can be problematic, as longer retention can be crucial for compliance, security investigations, and unforeseen circumstances. A forthcoming State of Observability report indicates that 94% of users consider long-term data retention important, despite the challenges posed by the cost of indexing and storage. Some organizations are addressing these costs by using data pipelines to route telemetry data to cold storage, but this can create data silos. Tools like Observe, built on Snowflake, offer a solution by allowing organizations to retain data for extended periods without incurring additional costs for rehydrating data from cold storage, thus breaking down silos and providing valuable context to the data.

Monitoring containers and Kubernetes requires a comprehensive approach due to the complexity and variety of data sources involved. Unlike traditional application stacks, containerized environments produce extensive monitoring data, including metrics and logs from containers, Kubernetes components, and the underlying infrastructure. The deployment context, such as using AWS's EKS, influences data collection strategies, as managed services provide different levels of control and observability compared to self-managed infrastructures. Additionally, the transient nature of container data necessitates real-time collection and external storage to prevent data loss, especially for logs that may not persist beyond container lifecycles. Container logs can reside in diverse locations, complicating the monitoring process and requiring flexible strategies to consolidate data from various systems, including cloud services like CloudWatch. Effective container monitoring involves using a mix of native tools and potentially automated solutions to aggregate and analyze data comprehensively, ensuring that observability remains centralized despite the distributed nature of containerized applications.