May 2023 Summaries

FlashAttention: Fast and memory-efficient exact attention with IO-Awareness` The proposed FlashAttention algorithm improves the efficiency of Transformers by making attention algorithms IO-aware, accounting for reads and writes between GPU high bandwidth memory (HBM) and on-chip SRAM. This reduces the number of memory accesses required by standard attention methods, resulting in a 15% end-to-end wall-clock speedup on BERT-large. FlashAttention enables longer context in Transformers, yielding higher quality models with better perplexity and accuracy, as well as entirely new capabilities such as achieving better-than-chance performance on long-range classification tasks. The algorithm's efficiency is further enhanced through network compression, reducing the required training time by 4x.

Together has secured a $20M seed funding round led by Lux Capital, supporting its mission to empower innovation and creativity through open-source generative AI models and a cloud platform. The company aims to provide leading alternatives to closed systems, enabling an open ecosystem for AI. With the backing of prominent investors including venture funds and entrepreneurs, Together plans to release projects that have garnered support from hundreds of thousands of AI developers, such as GPT-JT, OpenChatKit, and RedPajama. The platform leverages research in distributed optimization to efficiently scale training and inference, with plans to open up access to its cloud platform in the coming months. This marks the beginning of a new era of AI, where open models will give developers greater ability to understand, inspect, and utilize AI without vendor lock-in and with strong privacy protections.

The RedPajama-INCITE-3B model is an open-source, smaller foundation model with three key benefits: rapid iteration and experimentation, faster performance and scalability, and greater accessibility. The model can be run on commodity hardware, fine-tuned using LoRA, and used for few-shot prompts. The RedPajama-INCITE-3B model has been successfully used in several projects, including a legal Al startup and a social chatbot platform. It is designed to enable rapid iteration and experimentation, making it easier to improve models and downstream applications. The model's smaller size makes it more accessible to developers and researchers, allowing them to run or fine-tune foundation models on commodity hardware like laptops.

The RedPajama project aims to create a set of leading open-source models by rigorously understanding the ingredients that yield good performance. The project has released the RedPajama-INCITE family of models, including base, instruction-tuned, and chat models. The 3B model is the strongest in its class, with the small size making it extremely fast and accessible. The instruction-tuned versions achieve strong performance on HELM benchmarks. The 7B model outperforms the Pythia 7B model, demonstrating the value of a bigger dataset. The project plans to build models at larger scale using the new dataset, which will go beyond the quality of LLaMA 7B. These models are released under the Apache 2.0 license, allowing for use in both research and commercial applications.