May 2025 Summaries

San Francisco's favorite AI summit is coming back this Fall, Ray Summit 2025, taking place from November 3-5. The event will welcome distributed computing and machine learning enthusiasts to exchange ideas and share knowledge in a 3-day conference. To productionize AI applications, scaling systems and teams, and address real-world challenges is the focus of this year's summit. The Call for Proposals is open to all, including first-time speakers, with session lengths ranging from 30 minute breakout sessions to 15 minute lightning talks. Participants will be asked to select a talk track that aligns with their proposed session, covering topics such as Ray Ecosystem, AI/ML Applications, Research Frontiers, and Generative AI. The event aims to encourage the submission of proposals and provide guidance for first-time submitters, with accepted speakers receiving complimentary passes for the event.

Ray clusters are becoming increasingly important for scaling Python applications and organizations are looking for ways to deploy these clusters across different cloud service providers and neo-clouds. Anyscale is announcing first-class support for Azure Kubernetes Service (AKS) with advanced logging and monitoring, a Global Resource Scheduler (GRS) for capacity commitments, and upcoming multi-deployment management capabilities. This expansion completes Anyscale's native support across all three major cloud service providers - AWS, Google Cloud, and now Microsoft Azure, giving organizations unmatched flexibility to deploy AI workloads wherever it makes sense without the complexity of managing separate toolchains. The Global Resource Scheduler is designed to maximize resource utilization, enforce fairness with prioritization, and queue intelligently, ensuring critical production workloads never sit idle while development teams utilize available hardware during off-hours. Anyscale's upcoming Multi-Deployment support will allow customers to deploy resources across multiple providers, regions, and compute stacks within a single logical entity, providing seamless access to diverse cloud environments and compute stacks.

Ray Data has made significant improvements, including native join support via the `ds.join()` API, key-based repartitioning with `repartition(key=...)`, and a new custom aggregation API with `AggregateFnV2`. A hash-based shuffle backend powers joins, improving performance for repartitioning and aggregations, reducing memory pressure compared to previous implementations. The new backend works by hashing keys and partitioning data based on these hashes, allowing for more efficient shuffling and joining of records. Benchmarks show substantial improvements in runtime and reduced memory pressure for various workloads, including preprocessing, TPC-H Q1 SF100, and TPC-H Q1 SF1000. The hash-based shuffle backend also enables better performance for repartitioning and aggregations, reducing peak memory usage by up to 3.9x. Looking ahead, Ray Data plans to support different types of joins, logical plan optimizations, and further improvements for data preprocessors.

Anyscale's proprietary RayTurbo Data has been enhanced with significant improvements to transform how teams work with large-scale data, dramatically reducing both processing times and operational risks. These enhancements include job-level checkpointing to easily resume interrupted batch inference pipelines, vectorized aggregations to speed up computing statistics across large datasets, intelligent operator reordering with a focus on filter and projection operations. Combined, these can bring up to 5x speedup compared to open source Ray Data, making it ideal for competing in rapidly evolving markets where data processing at scale is crucial. Job-level checkpointing allows pipelines to resume precisely where they left off, reducing the restart penalty of failed jobs and minimizing wasted compute resources. Vectorized aggregations move computation from Python to optimized native code, eliminating performance penalties while maximizing throughput on modern CPU architectures. Intelligent operator reordering optimizes pipeline performance by pushing filters earlier in the execution plan and optimizing column selection. These improvements are designed to accelerate AI workflows, providing a competitive differentiator in today's fast-paced landscape where data processing at scale is essential.

The new Ray Train Dashboard and Ray Data Dashboard are two purpose-built observability dashboards designed to help ML engineers focus on model training and data processing logic, while providing a unified interface for accessing logs and metrics. The Ray Train Dashboard offers four critical observability features: training progress, error attribution, logs/metrics, and profiling, enabling users to visualize and understand what's happening in their distributed training jobs at different altitudes. It provides a unified experience, rich error context, one-click profiling, and workload-level abstractions to help identify performance bottlenecks and optimize performance. The Ray Data Dashboard integrates Tree and DAG views for pipeline drilldowns, operation-level metrics, and dataset-aware log aggregations, making it easier to quickly identify bottlenecks and optimize performance for data pipelines. Both dashboards are actively evolving to provide even more value, with planned enhancements including automated issue detection, integration with the Ray Train Dashboard, support for experiment tracking, and more.