April 2026 Summaries

SkyRL has introduced comprehensive support for vision-language model post-training, enabling teams to train multimodal models via both supervised fine-tuning and reinforcement learning workflows using scalable infrastructure. This platform supports a range of tasks requiring multi-step visual reasoning, such as computer use and robotics, by integrating vision-language models into the post-training stack. SkyRL facilitates the transition from supervised fine-tuning to agentic reinforcement learning for tasks involving complex visual environments, using tools like Tinker for recipe-driven fine-tuning and VisGym for multi-turn agentic reinforcement learning. The platform addresses challenges in aligning training and inference processes by implementing a disaggregated approach to ensure consistency and stability in model outputs. SkyRL also provides options for asynchronous execution and LoRA-based training, allowing scalability and efficient resource use, and invites community involvement to further enhance multimodal training capabilities.

Anyscale has announced the general availability of Anyscale Agent Skills, designed to enhance AI coding agents with specialized knowledge for writing, deploying, debugging, and optimizing workloads on the Ray platform. These skills are integrated into the Anyscale CLI and are available for tools like Claude Code and Cursor, supporting the full development lifecycle from generation to deployment and debugging. By utilizing Workload, Platform, and Infrastructure Skills, developers can achieve accelerated development speeds and ensure optimal performance of AI workloads. Additionally, Anyscale is launching an Optimization Services Program to identify and rectify performance bottlenecks and GPU inefficiencies in Ray workloads. This initiative aims to bridge the gap between general-purpose coding agents and the specialized requirements of distributed computing, offering a structured approach to deployment and debugging, thereby improving cost-efficiency and reliability in production environments. The skills also include robust security measures, and Anyscale's continuous development promises further enhancements and support for more AI tools in the future.

Ray Day Seattle, part of Anyscale's 2026 Ray on the Road series, showcased how companies like Notion, Salesforce, Uber, and Apple are leveraging the Ray framework to scale AI efficiently. At the event, Notion explained its migration from a Spark-based embedding pipeline to a streamlined Ray-powered job, achieving an 80% cost reduction and significant improvements in query latency. Salesforce discussed its document summarization pipeline using Ray, which processes up to 200K tokens with a P95 latency under 15 seconds, highlighting Ray's role in parallelizing tasks. Uber's presentation focused on enhancing GPU utilization and reducing training time by adopting Ray, while Apple demonstrated Ray's capability in handling massive foundation model training by unifying data processing and model training. The day also featured workshops for developers to deepen their understanding of Ray's core functionalities and its application in scalable data pipelines, distributed training, and production model serving.

The blog post discusses the implementation of DP Group Fault Tolerance in Ray Serve LLM for serving large sparse models, such as Mixture of Experts (MoE) models, using Wide Expert Parallelism (WideEP). This approach combines data-parallel attention with expert parallelism across multiple GPUs to improve memory efficiency, batch size, and throughput. In MoE architectures, expert layers are distributed across replicas within a Data Parallel (DP) group, requiring all participating ranks to be operational for successful query execution. Ray Serve LLM addresses the challenge of localized rank failures, which can disrupt the entire DP group, by leveraging gang scheduling primitives that allow for the atomic teardown and recreation of affected groups without disrupting healthy ones. This mechanism of fault tolerance ensures minimal service downtime, as traffic is rerouted to unaffected groups, enhancing the resilience of large-scale deployments. Additionally, gang-aware autoscaling maintains group integrity during scaling operations, ensuring that replicas are added or removed in full group increments. This method complements vLLM's Elastic Expert Parallelism, which enhances runtime elasticity within the engine, providing a robust framework for large-scale MoE model serving.