March 2024 Summaries

Fine-Tuned's latest edition highlights key developments in AI and LLM technology, including a new course on efficiently serving large language models (LLMs) by CTO Travis Addair, and significant platform optimizations that enhance fine-tuning speeds by up to 5x. The newsletter emphasizes the superiority of Upstage's Solar LLM, now available exclusively on Predibase, which outperforms GPT-4 on a majority of tasks. It features webinars, seminars, and workshops on fine-tuning open-source LLMs, illustrating the benefits of using technologies like Ludwig and LoRAX to address cost and latency issues in AI productionization. Additionally, it showcases a tutorial on fine-tuning Llama 3 for customer support and discusses Predibase's new fine-tuning stack, which, alongside specialized A100 clusters, offers unprecedented training speeds. The edition also includes insights from events like a webinar on accelerating LLM fine-tuning by 15x and a virtual workshop demonstrating how to fine-tune models to rival GPT-4, using efficient techniques and state-of-the-art optimizations.

Engineering teams are increasingly deploying multiple large language models (LLMs) in production, with nearly 70% of surveyed ML practitioners having already done so. Fine-tuning LLMs for specific tasks is identified as a cost-effective deployment strategy that requires access to advanced management tools. A major update to the Predibase platform now allows teams to manage LLM deployments through an enhanced user interface, offering features such as dedicated and serverless deployment options. Serverless deployments are cost-efficient for experimentation, as they charge per token and eliminate the need for idle GPU management. As usage scales, dedicated deployments become more appealing. The platform's new Deployments page offers a comprehensive view of all deployments, streamlining the monitoring and management process by displaying essential details like status and model context. The update also simplifies the creation of new deployments and provides detailed event histories and logs for better operational insight. Predibase aims to be the central hub for LLM management, offering enhanced visibility and control to facilitate rapid AI development and deployment. Users can start fine-tuning and serving models for free with an initial credit offering.

Recent advancements have significantly accelerated fine-tuning jobs, reducing processing time by 55-80% through optimizations such as shifting all SaaS customers to A100 GPUs and setting batch_size to Auto by default. An intuitive UI deployment page has been launched to manage serverless and dedicated deployments, offering features like event histories and detailed logs. Moreover, fine-tuned models can now be utilized if they pass a checkpoint, allowing teams to test mid-way and restart from checkpoints in case of failures, thus saving time and resources. Open Source LoRAX has been overhauled to prevent CUDA out of memory errors by automatically managing memory resources, alongside new enhancements in model architecture and batch size tuning for LLMs, which improve experimentation by enabling model weight loading from the latest training checkpoint.

Predibase Fine-Tuned highlights the company's advancements in fine-tuning open-source large language models (LLMs) for cost-effective and efficient AI applications, including the launch of "LoRA Land," which features 25+ fine-tuned Mistral-7b models that outperform GPT-4 on specific tasks. The recent events and resources shared by Predibase include webinars on fine-tuning models like Zephyr-7B for customer support and discussions on the future of fine-tuning using adapter-based training. Predibase has also introduced serverless solutions for fine-tuned LLMs, allowing users to pay-per-token, and offers resources such as a definitive guide to fine-tuning and tutorials for models like CodeLlama-70B-Instruct. Additionally, the company has joined the AI Alliance to support open innovation in AI and celebrated the Ludwig community's achievements, including a hackathon showcasing the potential of fine-tuning open-source LLMs.

LoRAX is an open-source inference server designed to serve large language models (LLMs) with support for multiple fine-tuned adapters on a single GPU, and its latest release, v0.8, offers native integration with the Outlines library for generating schema-compliant outputs. This is particularly useful for creating JSON outputs that adhere to specific schemas, which can be consumed by automated systems. The blog explores two core methods for generating JSON: structured generation and fine-tuning, demonstrating how each can enforce schema adherence or populate JSON with accurate content, respectively. By combining these methods, LoRAX achieves optimal results, producing outputs that are both structurally correct and content-accurate. A case study involving Named Entity Recognition (NER) tasks highlights the advantages of this combined approach, showing improved performance and reliability compared to using either method alone. The blog also addresses potential pitfalls, such as token limit issues and schema-model conflicts, underscoring the importance of aligning structured generation with model fine-tuning for optimal results.