June 2023 Summaries

whylogs has introduced UDFs (User-defined Functions) as a new feature to enhance data profiling. These custom metrics can be tailored to fit specific business or research objectives, making it easier to monitor complex data types such as text, images, and audio. With the release of whylogs 1.2.0, UDFs are available out-of-the-box for users to start using. The introduction of UDFs allows for standardization across teams and organizations, making it easy to define standards in a replicable manner.

WhyLabs has partnered with Databricks to enable seamless integration for monitoring AI pipelines. The WhyLabs AI Observatory is designed to monitor any scale of data in a distributed environment without the need to sample or move data around. This partnership allows for computation of all key telemetry data necessary for AI monitoring directly in Apache Spark, making it possible to monitor data quality at high speeds while maintaining full distribution. With WhyLabs, teams can answer questions about data of any size and gain visibility into their models' performance over time, as well as identify bias in predictions and correlate it to specific segments of the dataset. The integration is privacy-preserving and cost-effective, ensuring that data is handled with the highest level of privacy and confidentiality.

WhyLabs has been recognized as a leading AI company on CB Insights' 2023 AI100 list. The inclusion highlights the company's commitment to driving innovation and advancing the field of AI. As an observability platform, WhyLabs empowers high-performing teams to effectively manage and control the behavior of their AI and data applications. The company is working relentlessly to shape the tooling ecosystem and define the category of AI Observability.

LangKit is a solution developed by WhyLabs for understanding generative models like Large Language Models (LLMs). It enables users to monitor the behavior and performance of their LLMs, ensuring their reliability, safety, and effectiveness. With LangKit, AI practitioners can extract critical telemetry data from prompts and responses, which can be used to help direct the behavior of an LLM through better prompt engineering and systematically observe at scale. The tool allows users to establish thresholds and baselines for a range of activities such as malicious prompts, sensitive data leakage, toxicity, problematic topics, hallucinations, and jailbreak attempts. LangKit is simple and extensible, with the ability to extract all important telemetry about an LLM with just a prompt and a response. It also supports User Defined Functions (UDFs) for users to add their own metrics or validate prompts and responses in a particular way.

In today's data-driven world, ensuring the accuracy and consistency of large amounts of data is crucial for businesses. Data validation can be challenging as the volume of data grows. This tutorial introduces the concept of data logging and demonstrates how to validate data at scale using the whylogs open-source package. The case study focuses on validating Airbnb listing activity and metrics from Rio de Janeiro, Brazil. By leveraging data logging, businesses can generate statistical summaries of their data for monitoring, visualization, drift detection, and data validation purposes. Metric Constraints are a powerful feature built on top of whylogs profiles that enable users to quickly and easily validate the quality of their data.