February 2018 Summaries

In this talk, Josh Willis, an engineer at Slack and former Director of Data Science at Cloudera, discusses the importance of testing machine learning models in production. He emphasizes that deploying a machine learning model is not like deploying regular code patches and should be done repeatedly to ensure optimal performance. Willis also highlights the differences between lab data scientists and factory data scientists, noting their respective failure modes for machine learning. Additionally, he recommends reading "What's your ML test score? A rubric for production ML systems" as a guide for deploying machine learning models into production.

In a Test in Production MeetUp, Maria Verba, QA Automation Engineer at Handshake, discussed why and how her team tests in production. She emphasized the importance of having a healthy ecosystem for testing in production, including thorough tests, feature toggles, and monitoring. Maria also shared specific examples where testing in production is more beneficial than pre-production environments, such as performance testing with real-life traffic, using production data, and A/B testing and experimentation. Handshake uses LaunchDarkly to conduct experiments and A/B testing, which allows them to create records for additional conditions and configurations. The company plans to implement Canary testing in the future by diverting a portion of their traffic into new features.

In this talk, Cindy Sridharan discusses her thoughts on testing microservices in a more sane and efficient manner. She criticizes the industry's tendency to create incredibly complex systems for testing microservices, arguing that it contradicts the main purpose of using microservices - enabling teams to develop, deploy, and scale independently. Instead, Sridharan proposes thinking about testing as a spectrum rather than just something done during code writing or pre-production testing. She emphasizes the importance of coding for debugging, understanding operational semantics, and considering the boundaries between abstractions when building software. Additionally, she suggests that massive integration tests are an anti-pattern and proposes integrating with production environments to test single services.

In this episode of To Be Continuous, Edith and Paul discuss continuous integration with Microsoft program managers Simina Pasat and Joshua Weber. They talk about how Microsoft develops across platforms, uses feature flags, approaches user feedback, and more. The conversation also touches on the differences between traditional and non-traditional Microsoft developers, as well as the role of feature flags in enabling rapid delivery processes.