August 2021 Summaries

Machine learning models are becoming increasingly important in emerging products and technologies, transforming the process of building ML models from an art to an engineering practice. Ensuring a reliable experience for users is crucial when deploying models into production environments, where issues can have significant impacts on revenue and customer satisfaction. As the industry matures, reliability engineering has become essential to prevent model drift or failure, which can cause sudden and significant problems, such as plummeting sales and customer complaints. The importance of reliability engineering in ML initiatives cannot be overstated, requiring a structured approach to ensure successful model deployment and maintenance.

Operationalizing AI Ethics has become imperative due to the challenges posed by machine learning models aiming for real-life mirroring and prediction. Despite reputational, regulatory, and legal risks, many companies still lack the ability to identify, evaluate, and mitigate ethical risks associated with their AI/ML products. Reid Blackman suggests that implementing systems identifying ethical risks throughout an organization is crucial. His seven steps to operationalizing ethical AI include leveraging existing infrastructure, creating tailored risk frameworks, optimizing guidance for product managers, building organizational awareness, incentivizing employee involvement in risk identification, and monitoring impacts while engaging stakeholders. An approach focusing on integrating the most appropriate ML infrastructure tools and processes is recommended by Blackman to make AI socially and ethically responsible.

The article discusses the importance of best-of-breed ML monitoring and observability solutions in managing machine learning models. It emphasizes that model failure can have significant impacts on a company's revenue, public relations, and user safety. To handle these challenges, an effective ML observability and model monitoring platform is required to understand what's happening inside the model as it runs. The article also highlights two general types of machine learning monitoring and observability solutions: best-of-suite and best-of-breed platforms. While best-of-suite systems attempt to cover end-to-end visibility, they may not have a feature set to cover every possible use case. On the other hand, best-of-breed ML platform solutions focus on providing highly specialized tooling for specific use cases and can be interoperated together to take advantage of their strengths while avoiding their weaknesses. The article concludes by stating that best-of-breed monitoring and observability solutions are ideal for companies that take their AI investment seriously, as they provide valuable insights into the performance of machine learning models without requiring significant time or financial investments.

Data quality is crucial for machine learning (ML) systems as they rely heavily on data to function effectively. Poor data quality can lead to inaccurate model predictions, impacting the overall performance of ML models. As companies increasingly adopt ML technologies, ensuring high-quality data sources has become more important than ever. This article highlights the significance of monitoring and maintaining data quality throughout the entire process, from training to deployment.