The Future of Computing is Distributed

Distributed applications have been around for over 50 years but are becoming more prevalent due to the end of Moore's Law and the increasing computational demands of machine learning applications. The growth in these demands far exceeds the capabilities of even specialized processors, leading system architects to distribute computations across multiple nodes. Google and Nvidia have released distributed AI systems, while companies like OpenAI are using thousands of CPU cores for reinforcement learning simulations. To bridge this gap between application demands and hardware capabilities, new software tools, frameworks, and curricula are needed to empower developers to build such applications in the era of big data and AI.