Datalab: A Linter for ML Datasets

Datalab is an open-source platform that automatically detects common real-world issues in datasets, such as label errors, outliers, near duplicates, non-IID sampling, and low-quality/ambiguous examples, without requiring manual domain knowledge. It utilizes any trained Machine Learning model to diagnose dataset problems that can be fixed to produce a better version of this model. Datalab operates on predictions and/or representations from any ML model already trained, allowing data scientists to quickly analyze their dataset for issues and improve the quality of their data before training a new model. By automatically flagging data issues, Datalab enables data scientists to build reliable models from unreliable datasets, and its open-sourced nature makes it easy to add custom data quality checks or contribute to its development.