How Honeycomb Monitors Kubernetes

Honeycomb leverages its own platform to gain insights into its Kubernetes clusters managed through Amazon EKS, focusing on monitoring as a consumer of a managed service. One challenge addressed is detecting crashlooping pods, a difficult task with metrics-based approaches, which Honeycomb solves by using kspan to translate Kubernetes events into OpenTelemetry traces, enabling reliable detection and identification of problematic pods. Additionally, Honeycomb monitors AWS instance types to ensure optimal performance, utilizing the honeycomb-kubernetes-agent to detect when older instances are used, allowing for quick adjustments to prevent incidents. The OpenTelemetry Collector aids in monitoring the Horizontal Pod Autoscaler (HPA) by providing reliable metrics on desired pod counts, mitigating issues that arise from metrics spiking during deployments. Honeycomb's capabilities, especially in high-cardinality metrics support and event-based alerts, make it well-suited for Kubernetes monitoring, offering insights that facilitate proactive management and incident prevention.