May 2021 Summaries

The introduction of collective communication primitives in Ray 1.2.0 simplifies distributed operations by enabling processes to exchange information across many distributed processes simultaneously, significantly speeding up certain distributed operations. These primitives allow programs to express complex communication patterns between many processes, providing low-level control over the communication backend and optimizing for different types of computing devices. The `allreduce` primitive is a widely adopted collective communication primitive used in many distributed ML training systems, including Horovod and distributed TensorFlow. Ray's native collective communication primitives, such as `ray.util.collective.allreduce()`, can be used to simplify code and boost performance significantly compared to using the standard Ray APIs like `ray.get()` and `ray.put()`. The use of these primitives enables fast point-to-point communication between distributed GPUs and supports a variety of backends for collective communication.

The Ray Cluster is a distributed computing framework that enables efficient resource allocation by automatically scaling up and down based on an application's resource demands. It provides users with a serverless experience and can be used to speed up compute-intensive tasks, such as image processing, on large datasets. By using the Ray Autoscaler, developers can significantly reduce the complexity of their applications while minimizing infrastructure costs associated with running them. The Ray Cluster is open-source, compatible with all commonly used cloud providers, and does not create any vendor lock-in.

The Ray project has grown significantly since its inception at UC Berkeley, now having over 450 contributors from 100+ companies, with thousands of users adopting the framework for scaling applications. To guide their roadmap and improve product experience, the developers are conducting an annual community pulse survey to gather feedback on use cases, library integrations, tools, language(s), and OS, as well as understanding context and pain points. The survey aims to represent all users, including those using Ray with Java and C++ APIs, and will help prioritize features, abstractions, and APIs for optimal workflows. Completing the survey will also support a charitable cause, donating $2 per completion to selected organizations.

The Ray Community Pulse Survey, launched by the Ray project team, seeks to gather feedback from its users to guide future developments and enhancements of the Ray framework, which has seen significant growth since its inception at UC Berkeley. The survey aims to understand how users utilize Ray for scaling Python applications and libraries, including machine learning model training, serving, and non-ML workflows like data processing and ETL. With contributions from over 450 contributors across 100+ companies, Ray has integrated with libraries such as Horovod and XGBoost and improved features in Ray Serve and RLlib. To better prioritize features and address user pain points, the survey also explores the usage of new Java and C++ APIs, infrastructure setups, and cluster launch methods. As an incentive, the project offers a $2 donation to one of three charities for each completed survey, emphasizing the value of feedback from both new and long-term users to enhance the Ray experience.

Ray is a versatile platform for building diverse, computationally intensive applications, including data and AI applications and platforms, as well as long running applications like web services. It supports both business logic and model inference seamlessly, allowing developers to build and scale microservices as if they were a single Python application. Many companies have successfully used Ray for various applications, highlighting its flexibility and scalability. Ray also provides tools for testing, deployment, and monitoring, making it an ideal choice for building distributed applications that incorporate machine learning.