Home / Companies / Anyscale / Blog / Post Details
Content Deep Dive

Multi-model composition with Ray Serve deployment graphs

Blog post from Anyscale

Post Details
Company
Date Published
Author
Jiao Dong, Shreyas Krishnaswamy, Simon Mo, Edward Oakes
Word Count
2,554
Language
English
Hacker News Points
-
Summary

The new Deployment Graph API in Ray Serve enables fast local development to production deployment and is scalable with a unified DAG API across Ray libraries. It provides Python-native authoring experience, allows for easy composition of complex deployment graphs, and offers features such as independently scalable nodes, fractional resource allocation, shared memory, and dynamic dispatch. The API also supports function nodes, parallel calls, and optimization opportunities. With the Deployment Graph API, users can build, iterate, and deploy complex deployment graphs with ease, and Ray Serve will handle the serving aspect, including HTTP endpoint configuration and Python handle management. This new feature aims to simplify distributed computing and improve the overall developer experience.