May 2023 Summaries

The blog post discusses the enhancements made to the O11y GPT model to improve its ability to assist engineers and observability users by refining its approach to answering queries. The improvements involve fine-tuning the model with specific focus on OPAL query language, combining keyword matching with embeddings to create an optimal corpus, crafting prompts with role definitions and response boundaries, and maintaining context-specific history in conversations. The team has also integrated this technology internally to gather data on performance and user feedback, guiding further product development. Additionally, the post touches on the phenomenon of "hallucinations" in language models, which sometimes reveal system feature gaps that can be addressed. The technology is described as being in its early stages, yet promising in enhancing user interaction, with the potential for improvements and new features like generating OPAL code in worksheets through GPT-4. The post concludes with an invitation for users to try out the new GPT-powered integrations.

Observe uses OpenAI's large language models, specifically GPT-3.5, to enhance its O11y GPT Help bot, which assists users by providing succinct help and instructions based on their queries. The challenge lies in the model's token limitation of 4097, which restricts the amount of context and instructions that can be processed, especially given Observe's extensive 200,000-word documentation. To address this, Observe employs machine learning embeddings to efficiently map and retrieve relevant information from their vast documentation. They use the Ada-002 model to generate a 1536-dimensional vector for document segments, storing these in a FAISS index, which allows them to quickly identify and use the most pertinent document sections when answering user queries. Despite the model's wide-ranging pre-trained knowledge, it lacks specific updates on Observe's internal workings, leading to potential misinterpretations, especially where specialized syntax or rules are involved. To mitigate this, Observe simplifies its documentation into smaller sections, generates embeddings, and processes these efficiently, although unique challenges like syntax discrepancies still persist.

Observe, a company focused on observability, seeks to streamline the developer experience by leveraging modern technologies such as Snowflake to create a comprehensive event database that ingests various unstructured data types, allowing users to cross-reference metrics, logs, and traces without leaving the system. CEO Jeremy Burton, in a discussion with theCUBE's John Furrier and Dave Vellante, highlighted the shift from traditional enterprise software sales toward products that appeal directly to developers. Observe's un-opinionated approach enables it to handle both observability and security data, leading to the development of a new threat intelligence product that correlates internal log data with external threat intelligence feeds. While initially skeptical about AI's applicability in observability due to the rapidly changing nature of modern applications, Burton acknowledged the potential of technologies like GPT to enhance productivity by providing actionable insights and facilitating quicker onboarding with tools like their Opal language.

Observe anticipates that Large Language Models (LLMs) will become integral to many applications, leading to the emergence of "LLM Ops" roles to optimize user experiences. To support this, they introduced the OpenAI App, designed to enhance observability by providing packaged alerts, dashboards, models, and integrations. This app offers insights into system health, usage, and costs, but goes further by capturing detailed logs and timing information for each API request, thus delivering a comprehensive view of user interactions and performance. By integrating a few lines of code, developers can gain visibility into user queries and responses, helping to identify user confusion and improve the overall experience. The OpenAI App aims to facilitate better understanding and management of OpenAI integrations, enabling organizations to make informed, data-driven decisions to enhance their products.

Observe's new PostgreSQL App introduces advanced observability features for PostgreSQL databases, whether on-premises or cloud-based, by providing pre-built dashboards, datasets, and monitor templates designed to enhance database management and performance monitoring. The app integrates seamlessly with other data sources in the Observe ecosystem, such as AWS, Kubernetes, and OpenTelemetry, to offer a comprehensive view of infrastructure health and performance. Users can leverage the app's capabilities to address various operational questions, such as identifying performance bottlenecks, monitoring resource utilization anomalies, and optimizing configuration settings. The PostgreSQL App utilizes plugins like Telegraf and FluentBit to transform logs and metrics into actionable datasets, while its monitor templates provide alerts on critical metrics to ensure database reliability. Additionally, the app supports customization through user-generated dashboards and monitor templates, allowing tailored observability solutions. The app's integration into Observe's unique Observability Cloud architecture, featuring Data Apps, a Data Lake, and a Data Graph, facilitates linking and analyzing diverse datasets for broader insights. Users can install the app through detailed guides and benefit from usage-based pricing that supports extensive data ingestion without financial constraints.