March 2023 Summaries

The use of machine learning in everyday applications has raised concerns about the quality of training data, which is crucial for high-performing models. Data scientists spend a significant amount of time on tasks such as data selection, labeling, and evaluation to ensure high-quality outputs. To speed up the ML workflow, it's essential to choose the right data-centric tools, particularly for data labeling and inspection. Galileo automates data inspection by identifying low-quality data, reducing labeling costs by over 40% and improving model performance by over 20%. When used with Label Studio, a popular open-source labeling platform, Galileo can significantly improve the efficiency of building ML models, detecting annotation mistakes, and monitoring production data to train with next.

We inspect the ImageNet dataset, a popular computer vision dataset used today, and quickly find data quality errors while training a model. The dataset contains 1000 different classes with over 1.2 million samples, but it has rarely been updated since its release in 2012. We use Galileo to debug one of the most cited datasets in AI today and find tons of errors, including mislabeling of images such as 'tigers' as 'tiger cats'. The dataset's limitations are highlighted by our findings on class imbalance and the need for augmentation methods and more data to improve model performance. We also identify gaps in training datasets that may need to be patched before deploying a model in production. Our analysis shows that simple mistakes can have drastic effects on training and performance estimation, emphasizing the importance of data quality and robustness.