Serving four million Riffusion requests in two days

Riffusion`, a music generation tool, leveraged `Stable Diffusion` to create audio spectrograms that were interpreted as music. The project's popularity led to over four million song requests in two days, posing challenges such as handling immense traffic and optimizing cost. To address these issues, the team utilized caching model responses, scaling infrastructure beyond default limits, and leveraging a mixture of spot and on-demand instances to manage costs. These strategies enabled the successful deployment of Riffusion's high-traffic days while maintaining reasonable spend.