March 2021 Summaries

A machine learning toolbox is essential for teams to apply machine learning successfully in their products. The toolbox consists of three fundamental tools: a Feature Store, a Model Store, and an Evaluation Store. A Feature Store enables the centralization of feature transformations, allowing for offline and online serving, team collaboration, and version control. A Model Store serves as a central repository for models and model versions, enabling reproducibility, tracking lineage, and integration with other tools. An Evaluation Store provides performance metrics, monitoring, and evaluation capabilities to ensure model quality and continuous improvement. Additional tools like Data Annotation Platforms, Model Serving Platforms, and AI Orchestration Platforms can complement the toolbox by handling data annotation, model deployment, and workflow management, respectively. By leveraging these three core tools and additional complementary tools, teams can successfully apply machine learning in their products and achieve rapid innovation.

Tammy Le has joined Arize AI as Vice President of Marketing and Strategy, bringing her expertise in product marketing from roles at Atlassian and Adobe. As a self-proclaimed introvert who prefers written communication, Tammy developed a passion for storytelling through her work at Tubemogul, which was acquired by Adobe. With a background in psychology and a Professional Baking & Pastry Arts certification, Tammy is excited to bring her insatiable curiosity to Arize AI and nurture diverse perspectives in the team. She aims to define this emerging space in AI/ML and create simple and compelling product stories. In her free time, Tammy enjoys baking as a way to flex a different part of her brain.

Machine learning and artificial intelligence systems are increasingly used by major companies for critical business decisions, but they face challenges such as algorithmic bias. Companies like Apple have been accused of biased AI models, while the use of AI in the US judicial system raises concerns about perpetuating systemic biases. The responsibility for these issues often falls on users rather than creators, and society is struggling to catch up with the ethical implications of AI/ML technologies. To address this problem, steps include admitting the importance of ethical validation, making protected class data available to modelers, breaking down barriers between teams and data, and employing emerging technologies for accountability.

Model monitoring has become increasingly important as machine learning infrastructure matures. However, there is no foolproof playbook for measuring model performance in every situation. Performance analysis can be complex, especially when ground truth is not immediately available or biased. In such cases, proxy metrics and statistical distances can be used to monitor prediction drift. Additionally, measuring business outcomes alongside model metrics provides a comprehensive understanding of how models affect customers' experiences with the product.

Kunal Shah is the newest member of Arize's Front-end Engineering team. Kunal holds a bachelor's degree in Computer Science from the University of Southern California and has experience as a front-end engineer at Omada Health and Pandora. He was influenced by the tech scene in the Bay Area growing up and enjoys tinkering with mechanical components, having built computers for his family. Kunal is excited to join Arize AI and contribute to bridging the gap between humanity and technology, focusing on understanding ethics in AI/ML and mitigating risks for society. In his free time, he enjoys listening to music and maintains a blog at realhitsonly.com.