April 2024 Summaries

Arcee has introduced the ability to extract LoRA adapters from fine-tuned models using MergeKit, allowing for significant compression and resource efficiency. By comparing a fine-tuned model against a base model, differences in parameter values are decomposed into a low-rank adapter compatible with PEFT, effectively reducing the model's size from gigabytes to megabytes. This process not only facilitates easier sharing and merging of models but also enables the use of multiple fine-tunes during inference with lower resource demands. Experiments with OpenHermes 2.5 demonstrate that low-rank extractions retain most capabilities of full models, even achieving a 9.5% performance improvement over the base model with minimal parameter counts. This innovation offers solutions for fine-tuning challenges like catastrophic forgetting and supports multi-domain adaptation, paving the way for dynamic model merging tailored to specific requests.

Arcee has introduced a new feature in their MergeKit tool called Evolutionary Model Merging, inspired by Sakana.ai's recent work, to simplify the process of model merging by allowing users to target specific competencies or qualities in their merges. This technique replaces the traditionally manual and exploratory process of model merging with an optimization-driven approach, enabling users to specify desired model qualities and let the optimization handle the merge. The tutorial for Evolutionary Model Merging, using the feature flag "mergekit-evolve," includes setting up the environment, defining tasks with EleutherAI's language model evaluation harness, and writing a YAML configuration file for desired merge parameters. The method also allows the inclusion of custom tasks to cater to specific needs, like spatial reasoning or prompt format adherence. The process can be monitored in real-time using Weights & Biases, and the best merge configuration can be saved for further use. Arcee plans to integrate this functionality into its core product, offering a complete compute backend, thus eliminating the need for users to provide their own GPUs.

Domain knowledge probing, or Continual Pre-training (CPT), is a process used to integrate new knowledge into pre-trained Large Language Models (LLMs), typically requiring adjustments to a massive set of parameters. Although Parameter-Efficient Fine-Tuning (PEFT) techniques like QLoRA offer efficiencies by approximating necessary gradients with fewer parameters, they face limitations when applied to CPT. QLoRA is effective in instruction tuning and preference alignment, which involve smaller datasets focused on refining the model's existing capabilities rather than expanding them. However, QLoRA is inadequate for CPT, as it cannot introduce the significant new knowledge that CPT demands, as evidenced by research studies and experiments conducted by Arcee.ai using a Security and Exchange Commission dataset. The empirical evaluation showed that standard CPT outperforms QLoRA-based CPT, highlighting the latter's limitations in integrating extensive new knowledge into LLMs. While PEFT methods like QLoRA offer efficiency in specific tasks, they cannot replace CPT where extensive new knowledge integration is required, emphasizing the need for further research in developing more effective methods for continual pre-training.

Arcee, a leader in specialized language models for enterprise AI, is hosting a Model Merging Hackathon from April 19 to May 13, co-sponsored by AWS, offering $9,000 in cash prizes across categories like best new merge and best integration with other ecosystems. The event encourages participants to explore the innovative technique of model merging, which combines various fine-tuned large language models (LLMs) into a single, cost-effective model, reducing GPU dependency and enhancing performance. Arcee has partnered with Charles Goddard, creator of the MergeKit repository, to advance this field, and offers resources like the MergeKit GUI and tutorials to assist newcomers. The hackathon will be judged by MergeKit founder Goddard and Arcee CEO Mark McQuade, aiming to inspire creativity and fun in the model merging research community.

Meta has launched Llama-3, an advanced large language model, offering 8B and 70B parameter variants in both Instruct and Base formats, trained on 15 trillion tokens and refined with 10 million human-annotated samples. The Llama-3-70B model has achieved over 80% on the MMLU benchmark, making it the leading open large language model, while the Instruct variants demonstrate significant coding proficiency, scoring 62.2% and 81.7% on the HumanEval coding benchmark. It features a Tiktoken-based tokenizer with a 128k vocabulary size and a context window of 8,192, extendable if necessary, and employs various alignment techniques such as SFT, PPO, and DPO. Fully available for commercial use, Llama-3 is hosted on Hugging Face, and Arcee's MergeKit is being used to test the merging capabilities of this model, showcasing the seamless integration and adaptability of MergeKit with Llama-3 variants. The process involves setting up servers, authenticating with Hugging Face, and creating and running configurations for merging, ultimately enabling the creation and deployment of new merged models on the Hugging Face hub.

The Model-Tools repository on GitHub is a newly launched initiative aimed at the AI community, offering a collection of custom tools, scripts, and experiments designed to enhance, test, and simplify operations on Large Language Models (LLMs). The repository includes practical tools like a GUI frontend for the LLM Eval Harness and a Model Difference Examiner, which helps identify model shape and config mismatches, and an EasyPeftGUI for model adaptation. The tools are a result of addressing recurring inefficiencies in model analytics and modifications, with a focus on speculative optimizations and enhancements to model structures. Though currently featuring a limited number of entries, the repository is rapidly expanding, with ongoing refinement and documentation efforts. The project encourages community engagement and contributions, emphasizing the collective benefit of shared advancements in AI and open-source technology development.