August 2022 Summaries

We've been busy at Baseten, introducing new features such as a model deployment page and quality-of-life improvements to the process. Our model deployment now includes support for secret values directly in models run on Baseten, making it easier to access AWS resources, APIs, and databases securely. We've also introduced starter and business plans to give our power users the resources they need to scale. Additionally, we're hiring a lead ML engineer to build infrastructure and services for our users to deploy, serve, and monitor their ML models. Our newsletter will be back up in September.

Daniel Whitenack, Data Scientist at SIL, shares his expertise on building ML teams from the ground up, emphasizing the importance of gaining buy-in across the organization, proving value early through creativity and scrappiness, being a producer rather than a consumer with engineering teams, experimenting cross-functionally to identify highest impact areas, hiring owners over theorists, and overcoming failure by looping in end users early. He also highlights the advantages of startups' agility in leveraging pre-trained models and fine-tuning them for specific use cases.

As Nikhil Harithas, Machine Learning Engineer at Patreon, shares his experience of moving from software engineer to ML engineer, he emphasizes the importance of balancing speed with scalability in building an ML platform. He advises focusing on providing value early on, as fast as possible, and being willing to iterate quickly. When deciding whether to build or buy, Nikhil suggests considering if the problem is unique to the company, and prioritizing building solutions that provide the most value for the business or product. He also highlights the importance of reducing manual work and measuring problems better through a value-based prioritization framework. With his experience in operationalizing machine learning systems, Nikhil stresses the need to consider data collection, training, deployment, monitoring, and retraining when deploying models. When hiring next, he recommends prioritizing people with horizontal skill sets who can tackle various parts of the problem end-to-end. Ultimately, Nikhil believes that ML can be done at smaller companies with the right mentality, tools, and people, offering opportunities for increased ownership and direct impact on performance.

Truss is an open-source model serving solution designed to simplify the process of deploying machine learning models. It bridges the gap between model development and deployment by making it easy to serve a model on localhost and in production, reducing friction and accelerating machine learning productivity. Truss enables data scientists to build or deploy Docker images with a single command, package models in a standardized format, and share models within or beyond a team. The solution also abstracts away complexity from model serving, taking advantage of proven technologies like Docker, and provides features for iterative development, anomaly detection, and more. By open-sourcing Truss, the developers aim to participate in and give back to the ML community, provide transparency and confidence in their tooling, and show their work and hold themselves accountable to high standards for code quality and development practices.