February 2023 Summaries

Blue River Technology, a leader in autonomous vehicles for the agriculture industry, needed a robust and efficient solution to compute metrics on their autonomous tractors' performance criteria. The company's newly launched autonomous driving technology is improving farmers' productivity by enabling tractors to operate longer hours with limited supervision and in all weather conditions. To overcome the challenges of processing large amounts of data in real time, Blue River Technology turned to Anyscale, a platform that enables parallel processing at scale, which significantly reduced their regression test suite's processing time from 24 hours to less than 9.5 hours. The solution also provided persistent storage of converted bags and reports by using AWS S3 storage, eliminating single points of failure and making results more predictable.

The latest release of Ray 2.3 features significant improvements across the ecosystem, including enhanced observability, performance boosts, and support for new platforms such as ARM and Python 3.11. The dashboard UI has been restructured to improve usability, and a new timeline view allows for better debugging of errors in jobs. Additionally, the Ray Dataset Streaming feature, which improves efficiency and reduces configuration tuning issues, is now available in developer preview. Performance enhancements have also been made to reduce worker startup overhead, resulting in an 8X improvement. The library now supports the new gymnasium API and has added support for ARM64 architecture and Python 3.11. Ray Serve offers experimental support for multiple applications on a single cluster, allowing users to manage separate individual models and deployment graphs. With this release, the community is encouraged to provide feedback and contribute to future improvements.

Ray Summit 2022 highlights the use of Ray in training large language models, including OpenAI's GPT3, by companies such as Cohere.ai and UC Berkeley's Alpa project. Ray is used to automate model parallel training and serving, simplifying the coordination of distributing tasks across multiple hosts. The platform is also scalable, allowing for efficient training of large models like GPT3. Additionally, Ray is not just limited to language models, but can be used for generic distributed tasks as well. Companies such as Cohere.ai are using Ray on top of TPUs to distribute and schedule different tasks across many TPU hosts, while UC Berkeley's Alpa project uses Ray to automate model parallel training and serving of large language models like GPT3.

Ray is a unified framework that allows organizations to effortlessly scale their Python and AI workloads, providing an open and portable toolkit for the entire ML lifecycle and python native applications. Nixtla offers a set of libraries intended to make available comprehensive and performant forecasting capabilities in a simple and easy-to-use Python library. Nixtla's efficient algorithms leverage the computational scalability of Ray to enable easy forecasting at scale, with recent demonstrations showing the ability to train 1 million models in under 30 minutes. Anyscale provides a scalable infrastructure for time series modeling, allowing organizations to quickly allocate and downscale resources based on their forecasting cycles, providing accurate results in a timely and cost-effective manner. Together, Ray and Nixtla provide a powerful open-source solution for organizations looking to perform forecasting and anomaly detection at scale, with the potential to reduce errors by 20-50% and losses in sales and product unavailability by up to 65%.