August 2022 Summaries

PyTorch Lightning 1.6 was released with contributions from 99 developers across more than 750 commits, adding hardware support, distributed-training improvements, and reliability updates. The release introduces compatibility with Intel’s Habana framework and Gaudi AI processors through the HPU accelerator setting, enabling both single-device and distributed training workloads. It also adds the Bagua strategy, which provides advanced distributed training algorithms and may outperform standard PyTorch Distributed Data Parallel training. In addition, Lightning continues stabilizing its core Accelerator, Strategy, and Plugin APIs, including renaming TrainingTypePlugin to Strategy to better reflect its broader role; this change primarily affects users with custom accelerator, strategy, or plugin implementations.

PyTorch Lightning 1.7, developed through 492 commits from 106 contributors, expands distributed training, hardware compatibility, notebook workflows, and production-oriented tooling. Key additions include the experimental MPSAccelerator for GPU training on Apple Silicon, native PyTorch Fully Sharded Data Parallel support alongside the existing FairScale option, and Hivemind-based collaborative training across decentralized and potentially unreliable machines. The release also introduces experimental Distributed Data Parallel support in macOS and Linux Jupyter notebooks, addressing limitations of the previously available Data Parallel strategy. Other improvements include versioned and automatically reloadable “last” checkpoints, validation based on total batches across epochs, CPU and Habana device monitoring, automatic distributed sampler handling, inference-mode evaluation, warn-level deterministic operation support, and expanded LightningCLI capabilities. New extensibility features allow callback registration through Python entry points and simplified checkpoint customization, while experimental ServableModule validation and asynchronous checkpointing aim to help users prepare models for deployment and save checkpoints without interrupting training.

NVIDIA and Lightning AI introduced the beta Omniverse Replicator Lightning App at SIGGRAPH to help machine learning researchers create synthetic, physically accurate 3D training datasets without the expense and effort of collecting and annotating real-world images. Users can select built-in environments and assets or upload USD, STL, and OBJ files, then randomize object placement and rotation to generate varied rendered images with ground-truth bounding-box annotations for object-detection training. The app enables users to preview and refine datasets, generate samples, launch computer vision model training through Lightning with configurable hardware usage, monitor jobs, download checkpoints, and compare model performance on synthetic and real images. Available for free through a Lightning AI account, the beta also invites user feedback on features and desired library environments.

Lightning announced four senior executive hires to support its goal of improving interoperability and usability across the machine learning development lifecycle. Jatheen Anand, formerly of Veritas, Symantec, and Cohesity, joins as the company’s first VP of Engineering to scale products and teams; Priya Shivakumar, from Confluent, becomes VP of Product to develop differentiated tools and product strategies for a broad range of AI users; Mike Bilodeau, previously at Kong and Rubrik, joins as the first VP of Marketing to translate technical capabilities into customer-focused communication and go-to-market programs; and JP Hennessy, most recently involved in design initiatives at Meta, becomes VP of Design and User Experience to bring more intuitive, consumer-like experiences to AI development tools. The company presents these appointments as part of an effort to make advanced machine learning technology more accessible to researchers, enterprises, and other users while continuing to expand its community and workforce.