August 2024 Summaries

The blog post highlights various new features and enhancements introduced in Clarifai's platform, focusing on the Llama 3.1 model and its fine-tuning capabilities. Llama 3.1, developed by Meta AI, is a collection of pre-trained large language models available in three sizes, each optimized for different use cases. Clarifai offers the 8 billion parameter version, which can be fine-tuned using a dedicated template on their platform, allowing users to perform tasks like text generation and classification. The post also discusses the introduction of patch operations via the Python SDK for apps, datasets, input annotations, and concepts, enhancing flexibility in managing these elements. Improvements to the Hyperparameter Sweeps module and the face workflow are also covered, providing a more efficient way to optimize model settings and conduct face recognition tasks. Additionally, the post outlines restrictions on organization settings, improvements in SDK code snippet output, and the ability to delete associated model assets, reflecting the platform's commitment to user-friendly and secure operations.

The blog post explores the evolving landscape of multimodal datasets used to evaluate large language models (LLMs) that process multiple input types, such as text and images. It highlights various datasets and benchmarks, like TextVQA and DocVQA, which focus on optical character recognition (OCR) and visual question answering (VQA), and more specialized ones like MathVista for mathematical reasoning and LogicVista for logical reasoning. The post notes the trend toward creating curated collections of samples for comprehensive evaluation due to the growing number of datasets and the risk of model overfitting to specific benchmarks. It also mentions the importance of scalable infrastructure for deploying and running models, featuring Clarifai's Compute Orchestration as a solution for maintaining control over performance and costs across different cloud environments.

Large Language Models (LLMs) are increasingly being used to handle specialized domains like medical or legal fields by injecting domain-specific knowledge through techniques such as Retrieval-Augmented Generation (RAG) or fine-tuning. This blog post introduces and evaluates a fine-tuning method called Retrieval Augmented Fine-Tuning (RAFT), which enhances LLMs by leveraging generated Chain of Thought (CoT) responses to improve reasoning and answer generation capabilities in specialized domains. RAFT refines pre-trained models by generating high-quality CoT answers with a large model and then fine-tuning these answers on smaller, specialized models, bridging the gap between general-purpose LLMs and the specialized knowledge needed for specific fields. Experiments with models like Llama2-7B and Llama3-8B demonstrate significant performance improvements, with RAFT consistently outperforming RAG methods. Additionally, the method is efficient, requiring less data and computational resources, making it feasible for compute-constrained environments. The cost-effectiveness and scalability of RAFT suggest its potential for broader application, with ongoing evaluations exploring its performance on newer models and possible deployment on platforms like Clarifai.