Company
Date Published
Author
Qiao Han
Word count
579
Language
English
Hacker News points
None

Summary

BasisAI utilizes Grafana and Prometheus to monitor model drift in machine learning workloads by employing Boxkite, an open-source Python library that captures and compares data distribution shifts between training and production environments. By integrating with Prometheus, Boxkite generates histograms from both training and production features, which are then exported for Prometheus to scrape and subsequently visualized in Grafana. This setup allows for the detection and alerting of model drift, leveraging metrics such as KL divergence and K-S tests for quantifying drift in categorical and continuous data, respectively. The integration enables a unified approach to monitoring both ML and software performance, allowing teams already using Grafana to seamlessly incorporate ML monitoring through a JSON dashboard, while others can experiment locally with Docker.