July 2016 Summaries

Spike Aerospace, a pioneering engineering firm working on the first supersonic business jet with Quiet Supersonic Flight technology, has leveraged Rescale's cloud-based platform to efficiently run its computational fluid dynamics (CFD) simulations, thus avoiding the prohibitive costs associated with traditional on-premise infrastructure. By migrating its entire CFD process to the cloud, Spike Aerospace has achieved significant cost and time savings, enabling rapid product development without the need for substantial capital investment in hardware and software licenses. Rescale's platform, which integrates STAR-CCM+ software and provides scalable, pay-as-you-go HPC resources, has facilitated real-time global collaboration and enhanced data security, helping Spike Aerospace accelerate its supersonic jet development. The partnership with Rescale has allowed Spike Aerospace to focus on innovation and streamline its operations, supported by comprehensive product support and alignment with industry security standards.

The text draws an analogy between the naming conventions in the card game Magic: The Gathering and software engineering, emphasizing the importance of specific and evocative names. It explains how unique card names in the game allow for creative expansion without redundancy, which parallels the need for precise class names in software development to prevent large, unwieldy classes. Broad class names can lead to a clutter of responsibilities, making code difficult to read, modify, and prone to errors. By using descriptive and focused names, developers can create more manageable and understandable code structures, reducing cognitive load and fostering clearer logic distribution. The author, Adam McKenzie, emphasizes that thoughtful naming not only guides developers in assigning logic but also aids in the ongoing evolution of code.

The article introduces techniques for scaling deep neural network (DNN) training across multiple GPUs using the Torch machine learning library, which is favored by researchers due to its flexibility and open-source extensions that often house the latest deep learning advancements. It delves into two main parallelization strategies: Model Parallelism, where different parts of a network are processed by different GPUs, and Data Parallelism, where entire networks are run on different GPUs with various data batches. The article provides a simple Torch example to demonstrate these concepts and details how to use DataParallelTable for distributing data across multiple GPUs. It also references a practical implementation using Wide Residual Networks on CIFAR10, illustrating that multi-GPU setups can significantly reduce training time. The discussion is part of a broader series on multi-GPU and multi-node scaling, with future posts promised to explore other libraries and configurations.