May 2020 Summaries

The coronavirus pandemic has created an extreme environment that is challenging machine learning (ML) models trained on previously seen observations. Businesses with live production models are facing issues as these models make incorrect decisions based on data they have never encountered before. This article discusses the challenges faced by AI/ML models during such black swan events and provides best practices to build resilience in production AI/ML during outlier events and extreme environments. These include tracking and identifying outlier events, deciding on a model fallback plan, finding look-alike events, building a diverse portfolio of models, and understanding the uncertainty of model predictions when performance cannot be improved.

The machine learning workflow is broadly divided into three stages - data preparation, model building, and production. Model Building involves understanding business needs, feature exploration and selection, model management, experiment tracking, model evaluation, and pre-launch validation. Various ML Infrastructure companies offer platforms for different functions within the Model Building stage. Some of these include Alteryx/Feature Labs, Paxata(DataRobot), H20, SageMaker, DataRobot, Google Cloud ML, Microsoft ML, Weights and Biases, Comet ML, ML Flow, Domino, Tensorboard, Fiddler AI, Arize AI, and Stealth Startups. The challenges in Model Building include reproducibility of models, understanding model performance, and ensuring the model's performance in the experimental stage translates to real-world scenarios.

The text discusses the importance of Machine Learning (ML) Infrastructure platforms for businesses across various industries. It breaks down the ML workflow into three stages - data preparation, model building, and production. Data preparation is a crucial stage where raw data is transformed into inputs for training models. This involves sourcing data from different stores, ensuring completeness, adding labels, and transforming data to generate features. Various tools and platforms are available to assist in these tasks, such as Elastic Search, Hive, Qubole, Scale AI, Figure Eight, LabelBox, Amazon Sagemaker, Trifacta, Pixata, Alteryx, Spark, DataBricks, Domino, Databricks, Cloudera Workbench, and others. The text also highlights the challenges faced in data preparation, such as sourcing data from multiple locations, ensuring completeness, and maintaining clean data. It emphasizes the importance of tracking versioned data transformations and using feature stores to reduce duplicative work and compute costs.