September 2021 Summaries

The evolution of production machine learning architectures is similar to that of GPU programming architectures. The first generation was fixed function pipelines, the second generation introduced programmability within the pipeline, and the third generation aims to be fully programmable with a focus on libraries, making it easier for developers to use without worrying about the underlying architecture. Ray, a system being discussed as an example of a third-generation production ML architecture, has features such as simple and flexible framework for distributed computation, cloud-provider independent compute launcher/autoscaler, ecosystem of distributed computation libraries built with #1, and easy way to "glue" together distributed libraries in code that looks like normal python while still being performant. Ray is used by customers to simplify existing ML architectures, parallelize high performance ML systems, make machine learning accessible to non-specialists, and build highly scalable algorithms for deep reinforcement learning.

Ant Group has developed a serverless platform called Ant Ray Serving based on the distributed computing framework, Ray. The platform allows users to publish Java/Python code as online services and provides basic capabilities such as deployment, scaling, traffic routing, and monitoring. It supports 5 different usage scenarios within Ant and Alibaba and has reached a scale of 60,000 cores and 5,000 nodes. The platform is designed to help users focus on their own business logic while using Ray's distributed capabilities to develop online applications.

This article reviews common options for parallelizing Python code, including specialized libraries, process-based parallelism, and IPython Parallel. It highlights the benefits and drawbacks of each approach, providing code samples to illustrate their use. The article demonstrates how using NumPy can significantly improve performance in numerical computations, and how IPython Parallel and Ray can be used for interactive and distributed computing. Additionally, it touches on other Python implementations such as IronPython and Jython that offer multithreading capabilities. Overall, the article provides a comprehensive overview of parallelization techniques for Python code, highlighting their advantages and disadvantages, and providing practical examples to help developers get started with parallelizing their own code.