Announcing RayTurbo

Anyscale has introduced RayTurbo, an optimized runtime for Ray on its platform. The new offering aims to provide the best price-performance and developer capabilities for AI workloads compared with other solutions including running Ray in open source. Among other optimizations, RayTurbo reduces runtime duration of read-intensive data workloads by up to 4.5x compared to open source Ray on certain workloads, accelerates end-to-end scale-up time for Llama-3-70B by up to 4.5x compared to open-source Ray on certain workloads, and reduces LLM batch inference costs by up to 6x compared to repurposed online inference providers such AWS Bedrock and OpenAI. The platform is focused on four broad workloads in the AI development lifecycle: data processing, training, serving, and LLM workloads.