October 2022 Summaries

Recent machine-learning developments include Meta’s EnCodec neural audio codec for efficient high-fidelity streaming compression, DeepMind’s Algorithm Distillation method for in-context reinforcement learning without parameter updates, and DiffSBDD, a 3D generative approach to protein-targeted drug design. Shutterstock plans to offer DALL-E 2-generated images while exploring royalty payments for artists whose work contributes to AI training, and companies offering AI-based conversations with deceased people are raising consent and privacy questions. Bumble has open-sourced Private Detector, its tool for identifying and blurring unsolicited explicit images, while Microsoft released DeepSpeed-MII to facilitate low-latency, lower-cost inference for deep-learning models. The update also highlights Lockheed Martin and Red Hat’s work on AI-capable military systems, resources for distributed training and Lightning-based research projects, as well as upcoming AI conferences and paper submission deadlines.

Distributed training scales deep learning workloads across multiple GPUs or servers to address the memory and speed limitations of single-GPU training, particularly as datasets, batch sizes, and model sizes grow. The commonly used Distributed Data Parallel (DDP) approach replicates the model on each device, divides input data among them, computes losses and gradients independently, then averages gradients during backpropagation so every model copy updates identically. This can provide near-linear training-speed improvements as GPUs are added and is especially effective for large datasets and batches, but it does not solve cases where a model itself cannot fit on one GPU, which require sharding-oriented tools such as DeepSpeed. DDP also introduces concerns including duplicated samples when datasets cannot be evenly divided, possible inaccurate evaluation metrics, communication overhead from synchronizing metrics, and stalled processes, while frameworks such as Lightning automate much of the setup, workload handling, and failure detection.

Meta introduced a speech-to-speech translation system beginning with Hokkien to support primarily oral languages that have been underserved by text-based translation tools. Research highlights include Imagic, a diffusion-model method for complex text-directed edits to individual real images, an MIT technique for automatically differentiating programs with discrete randomness, and RARR, which researches and revises language-model outputs to reduce hallucinations and add sentence-level citations. The roundup also covers an AI-generated fictional Steve Jobs–Joe Rogan podcast, Denmark’s AI-guided Synthetic Party, and Pano AI’s expansion of computer-vision wildfire detection in Montana. Open-source developments include Google’s black-box optimization platform Vizier, Bluesky’s decentralized social-network beta, and Barilla’s Arduino-based device for lower-emission passive pasta cooking. It additionally promotes a Redis machine-learning tutorial, several Lightning and PyTorch community projects, workshops and livestreams, and upcoming deadlines and conferences including AAMAS, CVPR, IROS, NeurIPS, and the PyTorch Conference.

Lightning announced Muse, an open-source Stable Diffusion-based text-to-image generator accompanied by a tutorial for building diffusion-model cloud applications. Research highlights include DAAM, which uses cross-attention activations to map how prompt words affect generated image regions; LieGG, a method for extracting learned symmetry-group generators from neural networks; and DiL-piKL, which combines human data, imitation, planning, and reinforcement learning to improve cooperation with people in strategy games. Industry news covered Meta’s Quest Pro virtual-reality headset, the 2022 State of AI Report, and AWS’s Trainium-powered EC2 Trn1 training instances, while open-source releases included Google’s hardware- and framework-flexible OpenXLA compiler ecosystem and Microsoft’s FarmVibes.AI agricultural models. Community updates featured Hacktoberfest contribution opportunities, an AI pest-detection project for cotton farming, potential Lightning activities at NeurIPS, and submission deadlines and dates for major AI and machine-learning conferences.

Muse is an open-source text-to-image generator built with Stable Diffusion and presented as a blueprint for deploying diffusion-powered cloud applications through Lightning. Its initial implementation combined a React interface with a FastAPI model server hosted as a Lightning Work, allowing users to generate images at either faster 25-step settings or higher-quality 50-step settings. After the prototype encountered capacity limits at roughly 10 simultaneous users, its developers added a FastAPI load balancer, traffic-based autoscaling, and dynamic batching that groups queued requests for parallel GPU processing, increasing burst capacity from fewer than five users to more than 300 concurrent users. The project also integrates with Slack through a slash-command bot that sends prompts to the load balancer and posts generated images in channels. Lightning Apps manage infrastructure concerns such as cloud provisioning, GPU selection, storage, and parallel long-running workloads, while Muse’s open-source code can be cloned and adapted for other distributed AI applications.

An AI industry roundup highlights notable ICLR 2023 submissions, including DreamFusion for text-to-3D generation using pretrained 2D diffusion models, a quantum reinforcement learning proposal, and Phenaki for variable-length text-driven video generation. Research developments include Meta’s Make-A-Video system, AudioGen for text-guided audio synthesis, and ERNIE-ViL 2.0 for improved image-text pretraining, while engineering news covers the White House’s AI Bill of Rights blueprint, Oregon State’s Cassie robot setting a 100-meter running record, and Google Lens using GANs to make translated text blend naturally into images. Open-source updates feature Meta’s AITemplate inference system for AMD and NVIDIA GPUs, alongside a Microsoft warning about North Korean-backed attackers exploiting open-source software. The roundup also points readers to a PyTorch Apple Silicon GPU-training tutorial, community projects for image super-resolution and low-light enhancement, Lightning’s progress on PyTorch typing support, and upcoming submission deadlines for AISTATS, AAMAS, and CVPR.