May 2020 Summaries

In a podcast episode of The New Stack Context, Editorial Director Libby Clark interviews Christine Yen from Honeycomb, where they discuss Honeycomb's new pricing structure, insights from a survey on Observability, and the launch of an open-source collector for OpenTracing. This collector enables teams to import telemetry data from various open-source projects into any observability platform, including both their own and those of their competitors, fostering a more inclusive approach to data integration and analysis.

Danyel Fisher, a Principal Design Researcher at Honeycomb.io, provides insights on observability and critiques the overblown expectations surrounding AIOps in a feature for The New Stack. Fisher argues that while AIOps is often touted as a revolutionary solution for operational challenges, its current state does not fully deliver on these promises, failing to address fundamental issues in Operations. He explores both the potential benefits and the existing hurdles in system observation and service management, highlighting the gap between the hype and the actual capabilities of AIOps technologies.

Honeycomb emphasizes the importance of events as fundamental components for achieving observability in systems, defining an event as a "unit of work" that can vary in scope but generally manifests as either a trace span or a log event. For logs, the process of identifying events is straightforward, whereas for spans, Honeycomb equates one span to one event, enhancing system visibility through trace-aware instrumentation. Service owners can estimate Honeycomb usage based on their traffic patterns and the number of spans per service event, allowing a more precise calculation of observability needs. The process of instrumenting for high granularity initially is encouraged as it can reveal hidden bugs and inefficiencies, guiding further adjustments in instrumentation practices. Ultimately, the goal is to discern which traces are most valuable for ongoing monitoring and optimization, with Honeycomb providing support and resources for users to fine-tune their observability approaches.

Honeycomb released research findings indicating a growing trend towards adopting observability practices among engineering teams, with 80% intending to implement these practices within two years. The study, which surveyed over 400 engineering practitioners across various industries, reveals that while many teams are in the early stages of adoption or confuse observability with related tools like monitoring, a significant portion plans to reach advanced levels of practice soon. Advanced teams exhibit greater confidence in detecting and resolving production issues, with 90% setting and measuring service level indicators. These teams focus on outcomes such as higher-quality code and predictable release schedules, which contribute to reduced technical debt and improved system resilience. Conducted by ClearPath Strategies, the research underscores observability as essential for achieving production excellence and maintaining high-performing, user-centric systems.

Observability is a multifaceted concept that extends beyond mere tooling, emphasizing the importance of a mindset and clear processes in enhancing software development efficiency and user experience. As teams transition from legacy tools, they face challenges in adopting observability-driven development, which is hypothesized to improve efficiencies across the software engineering cycle and lead to Production Excellence. A study conducted with Clearpath Strategies, involving 406 respondents mostly in SRE/DevOps roles, reveals that while 80% of teams plan to adopt observability within two years, only a fraction currently falls into advanced categories. Many teams struggle with tool bloat, using multiple disparate tools that lead to confusion and inefficiency, though some, like Molly Struve, have successfully streamlined their tools to gain clarity and efficiency. The research identifies five groups on the observability spectrum, assessing practices like proactive bug detection and tech debt understanding, with advanced teams showing significantly higher confidence and capability. The journey to full observability adoption is iterative and requires a culture of shared ownership, with the promise of reduced toil and improved team satisfaction.