January 2022 Summaries

Ray Train is an easy-to-use library for distributed deep learning that aims to improve developer velocity, be production-ready, and come with built-in features. It simplifies the APIs of its ML ecosystem as it heads towards Ray 2.0. The library addresses the gap between prototyping and production model training by providing a framework that can bring the best of both worlds together - extremely fast iteration while making it really easy to scale on different cluster environments. Ray Train is designed for developer productivity, allowing developers to iterate quickly and easily integrate with third-party libraries. It provides features such as distributed data loading, hyperparameter tuning, built-in loggers, and support for PyTorch, TensorFlow, and Horovod. The library also offers a TrainingCallback interface that can be used to process intermediate results, making it easy to incorporate tools and utilities. Ray Train is open-source and flexible, allowing developers to leverage the open-source data ecosystem and integrate with various libraries and frameworks.

The Mars On Ray scientific computing framework combines XGBoost on Ray for end-to-end AI pipeline development. Mars is a unified tensor-based framework that scales NumPy, pandas, and scikit-learn functions, while Ray provides a simple, universal API for building distributed applications. The Mars On Ray architecture allows for seamless integration with Ray's large machine learning ecosystem, enabling fast and adaptive scale-out and scale-in, as well as automatic recovery from worker failures. XGBoost on Ray supports multi-node and multi-GPU parallel training, seamless integration with the popular distributed hyper-parameter tuning library Ray Tune, advanced fault tolerance mechanisms, loading distributed DataFrames as input data, and efficient data exchange through the shared memory object store. Mars On Ray is widely used at Ant Group and in the open source Mars Community, and its development continues to optimize performance, scalability, and efficiency.