November 2022 Summaries

JSON, originating from JavaScript as JavaScript Object Notation, is widely used for data interchange and contributes significantly to observability when passed between services, often appearing as a large string of attribute-value pairs. In the Observe platform, JSON data is typically ingested as objects in the FIELDS column and can be manipulated using OPAL, which treats extracted values as type "any," similar to Snowflake's VARIANT type. This flexible data type allows users to cast JSON values to specific types as needed, avoiding the complexity of parsing strings with regular expressions. JSON data can be extracted and parsed directly from larger text strings using functions like extract_regex() and parse_json(), enabling more efficient data handling and analysis in Observe. The platform encourages users to transform JSON into more readable forms and offers extensive documentation and support for those seeking more advanced guidance on working with JSON and OPAL.

OpenTelemetry (OTel) has emerged as a key player in the realm of observability, particularly in the context of complex microservice and Kubernetes-based applications. The project, under the Cloud Native Computing Foundation (CNCF), aims to standardize data collection across traces, metrics, and logs, addressing the challenge of telemetry data collection by offering a vendor-agnostic framework. OpenTelemetry is a result of merging OpenCensus and OpenTracing, leveraging the learnings from these projects to create a unified approach to telemetry. While it has made significant progress, particularly in the area of tracing, OpenTelemetry is still developing its capabilities in metrics and logging. Its extensibility allows for customization, but it also presents the risk of vendor lock-in if distributions prioritize specific functionalities. Despite challenges, the enthusiasm for OpenTelemetry is driven by its potential to simplify data instrumentation, making telemetry data more accessible and fostering a more ubiquitous observability framework that benefits engineering, DevOps, and SRE teams by democratizing data collection.

Observe has announced the launch of The Observability Cloud, marking a significant milestone in its five-year journey of developing a solution to address the complexities of troubleshooting distributed applications by eliminating data silos. The company recognized that the increasing volume of data required for troubleshooting has become economically unsustainable with traditional tools that charge based on data volume. Observe's new architecture leverages AWS S3 for data storage, elastic compute, and usage-based pricing to offer a more cost-effective solution, with benchmarks showing a 2-3x cost advantage over competitors. The platform supports the ingestion of event data into a unified Data Lake, enabling the curation of logs and metrics into a Data Graph for relevant context, with new features such as distributed tracing support via OpenTelemetry. Observability Cloud also offers visualizations, alerting, and pre-built Data Apps for services like AWS and Kubernetes, and emphasizes transparency and cost control for customers without requiring data filtering or sampling, ensuring efficient and economical troubleshooting capabilities.

Observe Direct introduces The Observability Cloud, a comprehensive solution aimed at consolidating data and eliminating silos to enhance troubleshooting efficiency. This new approach integrates various features like distributed tracing, cost management capabilities, and Observe Apps for platforms such as GCP, GitHub, and OpenTelemetry. Observe has seen significant growth, with over 60 customers and more than 800 monthly active users, who perform 40 million daily Snowflake queries. The Observability Cloud consists of a Data Lake, Data Graph, and Data Apps, enabling centralized data management and contextual analysis. Recent updates include new tracing visualizations without requiring OPAL, expanded app offerings, and enhanced cost management controls to provide users with transparency and budget adherence. The Metric Expression Builder has also been integrated into Dashboards, facilitating alert and monitor setup. Observe plans to expand its app ecosystem further, including an upcoming Azure App, reinforcing its commitment to evolving observability solutions.