September 2020 Summaries

Intelligent Medical Objects (IMO) leveraged Honeycomb's observability tools to enhance the performance and reliability of their systems by integrating tracing into their .NET application stack hosted on AWS. Faced with intermittent failures in their provisioning service, IMO discovered that their traditional reliance on unstructured logs obscured critical insights. By adopting structured logging and trace instrumentation, they uncovered performance issues linked to the use of AWSPowerShell modules, which incurred significant overhead during module imports. To mitigate this, IMO transitioned to using the AWS CLI, resulting in notable performance improvements, including reductions in execution time at both the 50th and 99th percentiles. This journey underscored the importance of building observability into system foundations, allowing IMO to identify and address architectural issues proactively, ultimately enhancing system efficiency and user satisfaction.

David Vellante, Co-CEO of SiliconANGLE Media, offers an analysis of the evolving landscape in application performance management, highlighting Honeycomb.io as a significant new player in the field. Described as a tool that advances beyond traditional monitoring, Honeycomb.io is positioned as an innovative solution for handling digital changes in this space.

Honeycomb is an observability tool that prioritizes using events over metrics due to the flexibility events provide in data analysis, allowing for arbitrary slicing and dicing of data to gain insights. Despite the limitations of metrics, such as their rigid pre-aggregation, they still offer valuable insights into system performance through measures like average latency and requests per second. Honeycomb supports both events and metrics, with the recommendation to replace most metrics with events while retaining essential infrastructure metrics related to system resources like CPU and memory. The "Getting Started With Honeycomb Metrics" whitepaper guides users on integrating metrics with Honeycomb, detailing the evolving role of metrics in observability systems and offering practical examples and code samples for sending metrics to Honeycomb. It emphasizes the importance of understanding the differences between events and metrics, providing tips for effectively analyzing both within Honeycomb's platform.

Emily Ashley, a community-taught software engineer, shares her experience of immersing herself in the world of software observability through the podcast o11ycast. Initially skeptical about her understanding of the topics, Emily quickly realized that the series offered insights into socio-technical systems and the human element behind technical challenges, resonating with her background in web cartography. By listening to diverse guests discuss real-world problems and solutions, Emily gained a deeper appreciation for observability as a paradigm shift rather than just a technical solution, understanding that it involves embracing system failures as learning opportunities. The podcast, hosted by Charity, Liz, and Adam, emphasizes the importance of viewing production environments as the true source of software understanding, advocating for continuous interrogation of systems to maintain adaptability and resilience. Emily found that this approach aligns closely with her experiences in smaller teams, where systems must be flexible and open to failure, and she encourages others to explore the podcast to see how observability can transform software development practices.

Honeycomb leverages its own platform to test and refine UX changes by using OpenTelemetry (OTel) alongside its existing Beeline SDK in its Poodle frontend service, which is responsible for rendering visualizations. Engineer Chris Toshok undertook the integration of OTel with Poodle to examine schema compatibility and performance differences, opting to double-send data to separate datasets to maintain existing functionalities and observe discrepancies. Toshok noted that Beeline's API is more closely aligned with Honeycomb's features, while OTel operates at a more general layer, lacking some Honeycomb-specific capabilities like trace-level attributes. Despite minor API differences and the need for type manipulation, the overall structure of instrumentation remained consistent, and performance remained unaffected after implementing OTel, highlighting the potential for seamless integration and evaluation of observability tools.