August 2022 Summaries

Artificial intelligence is revolutionizing healthcare with AI-focused startups raising over $12 billion last year. However, challenges persist in implementing AI at scale within the industry. Arize, an ML observability platform, recently received certifications from an independent auditor validating its health information security program's compliance with HIPAA Security Rule and the Health Information Technology for Economic and Clinical Health (HITECH) Act. These healthcare-specific certifications supplement Arize's broader SOC 2 Type II compliance. Arize is committed to safeguarding data, especially in healthcare, by focusing on auditability, prevention, and preparedness.

Sid Roy, Manager of Machine Learning Engineering at Devron, explains the concept of federated learning - a machine learning approach that enables training models on inaccessible data while preserving privacy. This technology is particularly useful for situations where companies want to access certain data but cannot due to privacy, regulatory, or jurisdictional reasons. Devron's platform allows data scientists to build, train, and evaluate machine learning models without ever having access to the data, making it a valuable tool in industries with strict privacy regulations. Roy also discusses the potential for federated learning applications in academia and how this technology can mitigate bias in models by improving the variety of data fed into them.

Arize has announced the next generation of machine learning (ML) monitoring to help teams scale their ML needs with increased automation, customizability, and flexibility. As AI technology evolves and matures across all industries, there is a growing need for more advanced ML monitoring systems that can accommodate various use cases and scale. Arize's new platform focuses on three major principles: automation with flexibility, programmatic monitoring access, and native alerting integrations. These features aim to make model monitoring automated and seamless, enabling users to identify and resolve issues faster.

Ray and Arize AI are two technologies that can help streamline the process of productionizing machine learning projects for scale and usability. Ray is an open-source distributed compute framework that enables users to run Python code in a parallel fashion across multiple machines, allowing them to focus on building their ML use case without getting sidetracked by managing distributed technologies. Arize AI is an ML observability platform that helps practitioners tackle issues such as model performance degradation, data drift, and data quality problems. Together, Ray and Arize can help teams scale the infrastructure around ML models while also improving team capabilities and allowing more time to be spent on building newer, better models for the business.

Tying model metrics to business KPIs upfront is paramount for ensuring alignment between ML and product teams. Investing all the way through the ML lifecycle is critical to ensuring AI ROI, as it requires planning for design aspects, human computer interaction, hypothesis development, and eventual retirement of models. Threading the needle with a centralized ML approach can be worth it, especially when blending product focus with broader technical breakthroughs. Assessing talent involves simulating real-world problems, such as giving candidates modeling tasks that tackle actual business needs, to understand how they think and approach challenges. By implementing these best practices, ML leaders can ensure a good foundation for future success in the rapidly evolving AI landscape.

Michael Stefferson, a Staff Machine Learning Engineer at Cerebral, discusses his transition from academia to industry and the key skills he developed along the way. He emphasizes the importance of understanding metrics, learning industry-specific systems, and being able to communicate effectively with both technical and non-technical stakeholders. Stefferson also shares insights on best practices for ensuring a model is ready for production, including establishing monitoring metrics and having contingency plans in place. Additionally, he highlights the challenges of working remotely and the importance of trust among team members.