Announcing Aviary: Open Source Multi-LLM Serving

Aviary is an open source project that simplifies the self-hosted serving of multiple LLM models efficiently, addressing issues such as cost, latency, transparency, deployment flexibility, data control, and customization. The project leverages Ray Serve to provide a highly flexible serving framework for scalable AI, offering pre-configured LLMs, acceleration approaches, simplified deployment, and autoscaling support. Aviary aims to bring these capabilities together in a convenient way for users, with plans for future expansion and community contributions. A hosted version of Aviary will also be offered, providing additional features around deployment and management, while existing Anyscale customers can access it at no additional charge via their workspaces solution.