January 2024 Summaries

In 2023, the artificial intelligence landscape experienced significant advancements, particularly with the emergence of OpenAI's ChatGPT, Meta's Llama-2, and a surge in open-source models that spurred innovation in generative AI. Small Language Models (SLMs) began driving enterprise AI adoption due to their cost-effectiveness and efficiency, while the "mixture of experts" model architecture enabled smaller models to outperform larger counterparts. Open-source solutions became more prominent as enterprises sought control over their AI models, moving away from costly proprietary APIs. The growing emphasis on synthetic data and modular architectures signaled a shift towards higher quality datasets and improved model reasoning, respectively. Efforts to combat LLM hallucinations by refining training techniques gained traction, while data-centric approaches emerged as pivotal in creating competitive moats for AI applications. The democratization of AI through open-source models improved transparency and accessibility, leading to broader adoption across industries. Additionally, there was a focus on integrating LLMs into software systems for machine consumption, and AI-optimized web browsing experiences began to take shape, promising dynamic and personalized online interactions.

In 2024, the focus is on making AI more accessible, reliable, and secure, with events and webinars dedicated to educating practitioners on leveraging open-source large language models (LLMs) for various applications, such as automating customer support tasks. The fine-tuning of models like Zephyr-7B and others is highlighted, demonstrating how open-source tools and methodologies can rival commercial options like GPT-4 while being cost-effective. The text also discusses community and open-source updates, including enhancements in AI tools like Ludwig and LoRAX, and emphasizes collaborations such as the AI Alliance to foster open innovation. Additionally, the launch of new features in Predibase’s UI aims to streamline the testing and iteration of fine-tuned models, further accelerating AI deployment and innovation.

Paradigm, a leading institutional liquidity network for cryptocurrency options trading, faced challenges with notification overload and missed opportunities due to a lack of personalized ranking in their Requests for Quotes (RFQs). To address this, they collaborated with Predibase to develop a deep learning-based recommendation system integrated with their Snowflake Data Cloud. Utilizing Predibase's ML infrastructure, Paradigm built predictive models that score maker-trade combinations, enabling real-time personalized notifications to market makers. This system improved trading volume by ensuring relevant market makers are alerted to crucial RFQs. Predibase's platform allowed Paradigm to efficiently develop, deploy, and maintain these models, significantly reducing the time and cost of production. The successful implementation not only enhanced trader engagement but also paved the way for Paradigm to explore additional machine learning applications, such as intelligent order book seeding and large language model fine-tuning, showcasing the adaptability of Predibase's technology.

Predibase's first edition of their newsletter, Fine-Tuned, reflects on a year of exciting developments in AI, while also providing insights into recent product updates and upcoming events. The newsletter highlights the introduction of support for fine-tuning and serving the Mixtral-8x7B model and outlines Predibase's aim to share best practices for building production AI, hands-on tutorials, and updates on their open-source projects Ludwig and LoRAX. Featured content includes webinars on fine-tuning open-source models like Zephyr-7B for customer support automation, leveraging LoRA for task-specific applications, and the benefits of adapter-based training. Predibase also emphasizes the potential for cost-effective deployment of AI systems using open-source models and introduces new features like a prompting experience in their UI and dedicated A100 capacity for training and serving models. The platform's updates include the release of LoRA Exchange (LoRAX) with expanded support for various models and quantization techniques, offering a comprehensive solution for efficiently fine-tuning and serving large language models.

Organizations are increasingly leveraging large language models (LLMs) to develop innovative internal applications, yet the high costs and slow speeds of these models have prompted a shift towards more efficient, distilled versions. The process of model distillation, which involves creating smaller, cost-effective models that retain the performance of larger ones, is gaining attention despite the challenges and guesswork involved. Drawing on experiences from Google and Predibase, a set of 12 best practices for LLM distillation is presented, using the Jigsaw toxic comment classification dataset as a case study. These practices aim to improve the efficiency and practicality of LLMs for developers and organizations seeking alternatives to models like OpenAI's GPT, which, while initially attractive due to ease of use and impressive performance, present issues such as high scaling costs and lack of ownership. The guide emphasizes the importance of quality teacher models, diverse and balanced datasets, starting with simple configurations, and monitoring models in production, while also exploring new techniques like parameter-efficient fine-tuning for efficient deployment. It encourages practitioners to adopt these strategies to optimize LLM development and deployment, contributing to the evolving landscape of open-source language models.

Leveraging open-source Large Language Models (LLMs) can significantly reduce the costs associated with customer support calls by automating the classification of customer issues, a task traditionally prone to errors and resource-intensive when performed manually. This tutorial demonstrates how to fine-tune an open-source LLM, specifically Zephyr-7B-Beta, to accurately predict Task Types from customer support requests, using a rich dataset of voice call transcripts. By employing tools like Ludwig, an open-source AI model framework, and Predibase, a managed AI platform, users can streamline the fine-tuning process on specialized datasets, resulting in improved accuracy and efficiency in identifying customer needs. These techniques not only enhance customer service by providing actionable insights for self-service improvements but also lower operational costs for support centers. The tutorial provides step-by-step instructions for setting up the environment, preparing data, and running experiments, showcasing that fine-tuned models can achieve a higher accuracy rate compared to the untrained models.