December 2022 Summaries

We are excited about the advancements in machine learning (ML) made in 2022, which have enabled rapid progress and adoption of foundational models. These models will likely empower more data scientists, ML practitioners, and developers to build ML-powered applications in 2023. The success story of Riffusion, a generative model for creating music, showcases the potential of these advancements, with its backend processing over 4 million song requests in just three months. Baseten has also introduced new features such as live reload with draft models, which accelerates testing and deployment processes by 100x, and Blueprint, a platform for building with generative AI, offering early access to subscribers on a waitlist. The team at Baseten looks forward to the growing energy in the ML space for 2023.

Riffusion`, a music generation tool, leveraged `Stable Diffusion` to create audio spectrograms that were interpreted as music. The project's popularity led to over four million song requests in two days, posing challenges such as handling immense traffic and optimizing cost. To address these issues, the team utilized caching model responses, scaling infrastructure beyond default limits, and leveraging a mixture of spot and on-demand instances to manage costs. These strategies enabled the successful deployment of Riffusion's high-traffic days while maintaining reasonable spend.

Baseten has introduced development deployments that unlock a live reload dev loop for ML model deployment, making testing updates take seconds instead of minutes, resulting in 100X faster dev loops. This allows data scientists and ML engineers to rapidly iterate on their models without breaking flow state, similar to web developers who enjoy instant feedback from their toolchains. With this innovation, Baseten users can accelerate their dev loop, test models in a production-like environment, and promote deployments to production with ease, saving time and resources in the process.