November 2021 Summaries

Superb AI and WhyLabs have partnered to provide data scientists and machine learning engineers with tools designed for their specific needs. Superb AI's platform enables the creation of high-quality training datasets through its customizable auto-label technology, while WhyLabs offers an AI observability platform that monitors ML deployments. Together, these platforms ensure reliable data operations by enabling users to identify and address issues such as data drift and model performance degradation.

Running and monitoring distributed ML systems can be challenging due to the need to manage multiple servers and different logs. However, Ray simplifies parallelizing Python processes, while whylogs enables users to monitor ML models in production even in a distributed environment. The key advantage of whylogs is its ability to operate on mergeable profiles that can be easily generated in distributed systems and collected into a single profile for analysis. This post explores options for integrating whylogs into Ray architectures as a monitoring solution.

The text discusses the importance of monitoring machine learning models to prevent performance degradation as real-world changes occur. It introduces WhyLabs, an AI observability platform that complements Amazon SageMaker for ML monitoring and observability. The blog post demonstrates how to use WhyLabs to identify training-serving skew in a computer vision example for a model trained and deployed with SageMaker. The ability of the whylogs library to extract features and metadata from images is highlighted, allowing users to profile based on images and understand differences between training data and serving data.

This article discusses the importance of improving observability for AI systems post-deployment. It presents an approach to enhance the observability of ML applications by efficiently logging and monitoring models using Flask and WhyLabs. The author demonstrates this through a Flask application for pattern recognition based on the Iris Dataset, integrated with the WhyLabs Observability Platform. The platform allows access to statistics, metrics, and performance data gathered from every part of the ML pipeline. The article also covers how to detect feature drift using monitoring dashboards.

Seattle-based WhyLabs has raised $10 million in a Series A funding round led by Defy Partners and Andrew Ng's AI Fund. The company provides an observability platform for monitoring data health and model health, which is now available for free self-serve usage. The investment will be used to meet growing demand, expand platform capabilities, scale operations, and continue building its world-class team. WhyLabs' AI Observatory enables teams to monitor, understand, and improve their AI applications, making it the first monitoring solution that AI builders can self-onboard and start using for free.