November 2022 Summaries

whylogs is a lightweight data profiling library that enables end-to-end data profiling across the entire software stack. It integrates with Apache Spark to achieve large scale data profiling and can be applied into existing data and ML pipelines. The integration is highly efficient, as it requires only a single pass of data and does not cause any shuffling. whylogs also supports both batch and streaming data sets, making it suitable for various deployment infrastructures. It provides a simple Spark API that can be used to extend the data set API and run various metadata and aggregation operations. The library is open source and has been designed with privacy, security, and compliance aspects of modern ML business requirements in mind.

In this article, the author demonstrates how to set up data logging and machine learning (ML) monitoring using open-source tools whylogs and WhyLabs. The process involves installing whylogs, importing necessary libraries, creating a dataset profile, setting up access keys for WhyLabs, writing profiles to the AI observatory platform, and enabling pre-configured monitors to detect anomalies in data quality, data drift, and model performance. Additionally, the author explains how to separate model inputs and outputs and monitor performance metrics such as accuracy and precision. The article also provides links to example notebooks for classification and regression monitoring on GitHub.

AIShield and WhyLabs are partnering to provide threat detection and monitoring for AI systems. AIShield offers a one-stop AI security solution, while WhyLabs provides AI observability. The integration of these two platforms allows users to prevent both AI attacks and failures, ensuring the reliability and security of their models. This partnership aims to address the growing need for comprehensive security insights into ML workloads in one place, bringing robustness against novel risks lurking in the present and immediate future.