October 2021 Summaries

Neptune AI and Arize AI have partnered to improve continuous monitoring and improvements for machine learning (ML) models. The collaboration aims to help ML teams maintain model performance once deployed in production environments, where data may shift in distribution or integrity due to unexpected dynamics or upstream changes. By connecting Arize's ML observability platform with Neptune's metadata store for MLOps, the partnership enables more effective monitoring of production models and informed retraining decisions. The integration allows users to track every model version and its history, improving experimentation and optimization processes in machine learning workflows.

There are significant financial losses due to global fraud, with the economy losing over $5 trillion annually. Building and deploying sophisticated machine learning (ML) models is crucial in detecting and preventing fraud, but these models can be fragile and require monitoring for anomalies. ML practitioners face challenges such as imbalanced datasets, misleading traditional evaluation metrics, limited sensitive features, and not all inferences weighted equally. To address these issues, important metrics to watch include recall, false negative rate, and false positive rate. Identifying the slices driving performance degradation is critical, and having an ML observability platform can help surface feature performance heatmaps to patch costly model exploits quickly. Additionally, monitoring and troubleshooting drift or distribution changes over time is essential in fraud models, as tactics are always evolving, and it's crucial to account for drift to ensure models stay relevant. By being proactive with monitoring and measuring drift, counter-abuse ML teams can get ahead of potential problems and focus energy on the most sophisticated threats.

Arize AI has been recognized by Gartner as a Cool Vendor in its Enterprise AI Operationalization and Engineering report. The platform helps address three key challenges inhibiting AI operationalization, including automatic detection of problems such as data quality or drift issues, faster root cause analysis and problem resolution of ML models, and continuous improvement of model performance, interpretability, and readiness. Arize is a machine learning observability platform that assists ML practitioners in taking models from research to production with ease. The recognition underscores the importance of ML observability as a critical category in ML infrastructure and positions Arize as a leading pioneer in this emerging space.

The role of machine learning (ML) engineers at Chick-fil-A involves building and scaling analytic capabilities to support business strategies, delivering continuous value for the company through innovative solutions. Korri Jones, senior lead ML engineer, emphasizes the importance of hiring tenacious thinkers with big hearts and a natural curiosity, as well as having a shared ownership approach across teams to achieve goals. The organization prioritizes bridging the gap between data scientists and data engineers by providing the right tools and technology to achieve scale and performance without losing velocity. Jones also stresses the need for leadership awareness, support, and understanding of ML initiatives to drive success, citing Chick-fil-A's leadership as a key factor in their growth and innovation. The ultimate goal is to deliver unique value that empowers owner/operators to make an impact in their communities and provide exceptional customer experiences.

Amber Roberts, an astrophysicist and self-taught data scientist, has joined Arize as a Machine Learning (ML) Sales Engineer. She was drawn to Arize due to its focus on using ML as a powerful force for change in businesses, the economy, and society. Amber's background in astrophysics and her passion for solving big problems made her an ideal candidate for this role. In her new position, she will help address pressing issues such as bias, responsible AI, ML observability, and explainability. As a sales engineer, Amber will be crucial in understanding the technology, business case, and varying levels of complexity based on the organization's ML infrastructure.

The last decade has seen a surge in interest in machine learning, with numerous researchers attempting to solve complex problems using state-of-the-art techniques. This renewed interest has led to an explosion of applications using machine learning to deliver novel experiences. However, as these applications move from research labs to production environments, new challenges have emerged that must be addressed in order for the ML systems to succeed. In order to measure and improve service-level performance, it is no longer sufficient to only monitor data quality or system performance over time; rather, overall service performance must also be evaluated.