March 2017 Summaries

IBM Bluemix, in collaboration with Rescale, offers a streamlined and user-friendly cloud-based platform for high-performance computing (HPC), which is especially beneficial for professionals like data scientists and aerospace manufacturers who may not be HPC experts. Rescale's ScaleX platform simplifies the complex tasks of HPC configuration and optimization, allowing users to run sophisticated computations on IBM Bluemix infrastructure with ease. This collaboration provides automated cluster configuration, pre-tuned software applications, and cloud management features that enhance performance and facilitate hybrid cloud deployment. The integration also includes collaboration tools for real-time sharing and management capabilities for enterprises, making it a cost-effective and secure solution for scaling computations. The partnership aims to leverage these synergies to expand the utilization of IBM's HPC infrastructure, with future developments planned to enhance capabilities further.

This article explores multi-GPU training with TensorFlow, focusing on data-parallel GPU training in both single and multi-node configurations using Rescale's infrastructure. It starts with the preparation of datasets, converting images into TFRecords format using smaller datasets like Flowers before scaling to larger datasets such as ImageNet. The training uses the Inception v3 deep neural network architecture, leveraging Rescale's MPI-configured clusters to manage distributed training processes, which include GPU-based model training and CPU-based model evaluation, alongside visualization using TensorBoard. The article details the creation of Rescale jobs for single-node and multi-node configurations, utilizing MPI scripts and TensorFlow's distributed training capabilities, to efficiently manage GPU resources across multiple nodes. It concludes with a brief mention of upcoming discussions on the performance implications of distributed training across various server configurations.