January 2023 Summaries

Recent AI research highlights include a University of Maryland framework for watermarking proprietary language-model outputs without API or parameter access, StyleGAN-T for fast, high-quality text-to-image generation, and Meta’s Multiview Compressive Coding approach for predicting 3D scene structure from images using RGB-D training data. Industry updates cover Shutterstock’s multilingual AI image generator, the proposed $2.6 billion U.S. National Artificial Intelligence Research Resource, and Zoom’s conversational Virtual Agent for customer support. Open-source releases include OpenAssistant and trlX implementations of ChatGPT-style training, while community and Lightning AI resources feature Stable Diffusion serving optimization, project templates, keypoint detection, fine-tuning tools, educational materials, cloud credits, and upcoming submission deadlines for SIGIR, KDD, IROS, and ICCV 2023.

Stable Diffusion serving can be made substantially faster and less costly for high-concurrency production workloads through PyTorch and DeepSpeed optimizations while preserving image quality. Tests on an NVIDIA A10 GPU reduced single-image batch-size-one serving time from 6.4 seconds to 2.09 seconds, an improvement of more than 300%, compared with much slower results on M1 Mac hardware. Mixed-precision FP16 inference produced a 40% speed gain, while inference_mode or no_grad offered less than 1%, and CUDA Graphs added about 5% by capturing and replaying GPU operations to reduce Python and kernel-launch overhead; CUDA Graphs were applied to the CLIP text encoder, U-Net, and VAE but do not support dynamic batch sizes. DeepSpeed Inference yielded a further 44% gain through fused GPU kernels and optimized transformer layers, requiring custom replacements for Stable Diffusion’s CrossAttention and BasicTransformerBlock components because the model was used directly from its repository rather than through pre-supported Hugging Face or Timm implementations. The optimizations also delivered stronger gains at larger batch sizes, and future work may investigate dynamic batching for the U-Net and operator trace optimization.

Recent machine learning developments include DeepMind’s Tracr compiler, which translates human-readable RASP programs into weights for GPT-like transformer models to support interpretability research; a cognitive-science analysis finding that large language models show strong formal linguistic abilities but remain limited in reasoning, mathematics, and general knowledge; and Microsoft’s GLIGEN method for generating images conditioned on captions and spatial grounding inputs. Engineering news highlights GPTZero, a student-built tool that estimates whether writing may be AI-generated using text perplexity, Airbus tests of computer-vision-supported landing, diversion, and taxi technologies, and Google Cloud’s image-based retail shelf-tracking system. Open-source work includes RLGym, a reinforcement-learning toolkit behind Rocket League bots capable of competing with elite players, while a Lightning tutorial explains training a 20-billion-parameter GPT model in the cloud. The update also announces Lightning 1.9 stability and distributed-training improvements, revived community forums, several community code contributions, and submission deadlines for ACL, ICML, IROS, and ICCV 2023.

Lightning is a platform from the creators of PyTorch Lightning that extends streamlined model training into serving, deployment, monitoring, and data engineering for production machine learning workflows. Built around the open-source Lightning App Framework, it lets users run Python applications locally, on AWS-based Lightning Cloud, or on private clusters with minimal infrastructure management, while allowing configurable compute resources such as CPUs, GPUs, RAM, storage, and software dependencies. Its architecture uses LightningWork components for long-running tasks including training, inference serving, ETL, and interactive demos, while LightningFlow coordinates multiple Works and defines application logic, such as training a model before deploying it. The platform supports separating hardware choices across pipeline stages, enabling models to train on GPUs and serve on CPU machines, and applications can be launched locally or in the cloud through simple command-line options. As machine learning systems grow, teams can add capabilities such as drift detection, monitoring, autoscaling, dynamic batching, and hyperparameter optimization to create more complete MLOps pipelines.

Machine learning teams may need cloud infrastructure as datasets, checkpoints, and logs grow, but managing cloud resources can add operational complexity. Using the Iris dataset, the walkthrough first demonstrates conventional Scikit-learn decision-tree training, evaluation, and visualization, then shows how to migrate the workflow into a LightningWork component. Lightning’s CloudCompute configuration allows users to select resources such as CPU and disk capacity, while its Drive API provides persistent shared storage for saved model checkpoints that can later support deployment or other workflow components. The application can run locally or in the cloud through Lightning commands, with training logs, code, artifacts, and downloadable models available through the Lightning AI dashboard. The approach can also support scheduled or condition-triggered training, deployment, and cloud-based hyperparameter optimization, while existing Scikit-learn scripts can generally be migrated by placing configuration in a LightningWork initializer and training logic in its run method.

Highlights include DeepMind’s DreamerV3 world-model algorithm, which reportedly achieves strong cross-domain performance and can collect diamonds in Minecraft without human data, Stanford’s LILAC framework for incorporating natural-language corrections into task execution, and Google research questioning whether causal tracing reliably identifies the best locations for editing facts in language models. Engineering news covers a programmer’s virtual AI companion built with ChatGPT and generative-media tools, OpenAI’s indication that ChatGPT monetization is under consideration, and Bird Buddy’s camera-equipped hummingbird feeder capable of identifying hundreds of species. NASA is seeking input on which open data, software, and computing resources would most benefit researchers, while a tutorial explains how to move Scikit-learn training from local machines to cloud infrastructure as workloads scale. Additional updates include Lightning community contributions, techniques for pairing Lightning, Stable Diffusion, and GPT-3, refreshed Lightning forums, and upcoming 2023 submission deadlines for ACL, ICML, IROS, and ICCV.

Recent AI developments include Google’s Muse text-to-image model, which uses masked generative transformers and discrete tokens to improve efficiency over diffusion-based systems, SparseGPT, a pruning method reported to reduce large language models to 50% sparsity without retraining, and rethinking with retrieval, a post-processing approach intended to improve LLM reasoning without fine-tuning. Microsoft is considering integrating ChatGPT technology into Bing to provide more natural and reliable search responses, while Axelera AI has introduced compact hardware accelerators for machine-learning workloads. DoNotPay plans to test an AI assistant that listens to traffic-court proceedings and relays guidance to a defendant through an earpiece. The update also promotes a Lightning webinar on using dynamic batching and autoscaling to serve Stable Diffusion to more than 1,000 concurrent users, recognizes community contributions to Lightning documentation and logging support, highlights a Harvard AI course book, and notes upcoming 2023 submission deadlines for major AI, machine learning, robotics, computer vision, and language conferences.