July 2022 Summaries

Validating data quality is crucial for machine learning applications as poor data quality can lead to pipeline and model failure. This post explores why validating data quality is essential in the MLOps process and how to use the open-source whylogs library to perform data quality monitoring in a Python environment. The techniques discussed can be applied to any application that uses a data pipeline. Data quality validation ensures data is structured and falls within the expected range for pipelines or applications, preventing unwanted machine learning behavior in production. Using whylogs, an open-source data logging library, users can create lightweight profiles containing statistical summaries of data for measuring data quality, drift, and model drift in any Python environment. The feature in whylogs for performing data validation is called constraints, which can be set on default profile metrics or user-defined custom metrics.

The WhyLabs engineering team improved SQLite performance by making monitoring data and machine learning models faster and easier for whylogs users. They increased transactions per second (TPS) and p95 latency by orders of magnitude through some small, obscure changes to the SQLite code. These improvements included omitting vacuums, reusing a single database connection for the life of the container, handling disabling autocommits, and committing at the end of blocks. The new code performs significantly better than the old one, with roughly 5 times faster TPS and a p95 request time that is consistently fast enough for users to monitor their data & machine learning models easily and quickly.

The open-source whylogs profile visualizer enables visual observability of data in machine learning and data engineering with Python. It helps to spot differences in the distributions of two data profiles at a glance, quickly pinpoint data drift in your machine learning operations, and maintain the quality and relevance of high-performing data and machine learning models. The profile visualizer runs right in a Jupyter Notebook along with your logging code and grows the interpretative capabilities of the open-source whylogs library quickly becoming the standard for data logging and monitoring of ML models and data pipelines.