November 2022 Summaries

:` Spark New Zealand, a telecommunications company, has been leveraging machine learning (ML) to improve its customer experience, business processes, and overall competitiveness in the market. The company's ML team, led by Habib Baluwala and Aadil Dowlut, has developed over 50 models in production, with a focus on improving marketing efficiency, predicting churn, and optimizing business processes. Spark New Zealand uses an ML observability platform like Arize to monitor model performance, detect drift, and provide insights into fairness and bias. The company's approach to building versus buying an ML observability solution was initially considered internally but ultimately led to the adoption of a commercial platform due to its ease of use, speed, and reliability. With Arize, Spark New Zealand has been able to speed up time-to-resolution for model performance issues and improve model improvement by better understanding and addressing customer behavior. The company's leadership prioritizes ML observability as a core part of its business, and the MLOps pipeline is at the heart of model development and automation. For those entering their first ML or data science role, Habib Baluwala advises taking time to understand the fundamentals of machine learning, understanding the business problem first, and not skipping any steps in the data science process.

Xander Song, Arize's new Developer Advocate, brings an interdisciplinary background and experience as a machine learning engineer at Test.ai to his role. He is interested in early-stage companies at the intersection of AI and ops and aims to shape processes, techniques, and best practices that define a developer's workflow. As a former researcher who transitioned into machine learning engineering, Song understands the importance of innovation in modern AI and ML, including the development of tools that support model performance. At Arize, he will focus on evangelism and advocacy for machine learning observability, emphasizing its role in improving quality of life for MLEs and data scientists while potentially making a difference between enterprise success and failure. Song brings a customer-obsessed mindset to his new role, having worked with startups and teams that prioritize user experience and value proposition.

This blog post focuses on developing an image classification model using the Fashion MNIST dataset and monitoring its performance over time by analyzing embedding vectors associated with input images. The authors provide a step-by-step guide to automatically surface and troubleshoot the reason for performance degradation, including data preprocessing, training, extracting image vectors and predictions, logging inferences into the Arize Platform, and preparing data to be sent for monitoring. The post highlights the importance of monitoring model performance, especially in industries like healthcare or self-driving cars where safety is paramount, and provides a robust and automated way to stay on top of model performance using tools like Arize.

Shafiq Shivji, Group Product Marketing Manager at mParticle, discusses the importance of real-time data pipelines in an interview. He explains that a customer data platform (CDP) like mParticle simplifies data ingestion, unification, and activation by providing easy ways to manage data pipelines and stream data to downstream destinations. CDPs enable real-time personalization use cases without requiring heavy engineering lifts to instrument and maintain data pipelines. Common use cases include marketing, product management, analytics, growth, and ML modeling. Shivji highlights the difficulty of building data-quality pipelines due to constant change in technology and business realities, and emphasizes the importance of personalization for enhancing user experience and driving sales, loyalty, brand, and customer satisfaction.

Monitoring ranking models is crucial for ensuring high-quality recommendations and maintaining customer satisfaction. Poorly performing ranking models can lead to decreased revenue, increased churn, and reduced user engagement. To monitor these models effectively, it's essential to use rank-aware evaluation metrics such as Mean Reciprocal Rank (MRR), Mean Average Precision (MAP), and Normalized Discounted Cumulative Gain (nDCG). These metrics help gauge the relevancy of predictions and their order. By leveraging machine learning observability, companies can proactively identify performance degradation, uncover the worst-performing features and slices, and quickly root cause model issues to improve overall ranking model performance.