November 2024 Summaries

Fine-tuning LLMs for multi-turn conversations involves adapting open models to specific business contexts, addressing challenges such as domain adaptation, knowledge constraints, and maintaining context across multiple exchanges. This process requires a smaller, high-quality labeled dataset of domain-specific examples. Multi-turn fine-tuning helps models handle domain-specific queries with greater accuracy and ensures they respect unique guardrails in business contexts. Dataset preparation is crucial for successful fine-tuning, ensuring proper conversation structure, clear turn delineation, system messages to set the context, consistent role labeling, and JSONL format compatibility. Loss masking in instruction fine-tuning refers to selectively including or excluding certain parts of input when computing training loss, with three approaches: no instruction masking, full instruction masking, and boilerplate masking. Recent research suggests that not masking instructions often leads to better model performance compared to the traditional approach. Fine-tuning LLMs for multi-turn conversations requires careful attention to dataset preparation, training implementation, and evaluation, with optimal results achieved by starting with high-quality conversation data, proper input masking, using parameter-efficient fine-tuning methods, and monitoring and evaluating throughout the process.

Long context fine-tuning has become increasingly important as large language models (LLMs) can now handle millions of tokens, enabling enterprise applications such as document analysis and summarization systems. However, implementing reliable long context LLMs is challenging due to performance degradation for context length thresholds much smaller than the maximum limit. Fine-tuning smaller models on longer contexts can enhance their performance at a fraction of the cost. The Together AI platform now supports fine-tuning on context lengths up to 32k tokens, with plans for even longer sequence lengths. This approach is particularly valuable for enterprise applications where data privacy and ownership are crucial considerations.

The Fine-tuning API has introduced new features, including long-context training, conversation data support, and more configuration options. These updates aim to enhance the performance of specific tasks by allowing ML teams to customize open models easily. Longer-context fine-tuning supports up to 32K context length for Llama 3.1 8B and 70B fine-tuning and inference, while conversation and instruction data format support streamline data preparation. Training quality improvements have been made without any changes in hyperparameters, inputs, or cost of fine-tuning jobs. Validation dataset support allows users to monitor the loss of the model on unseen data during training. Quality-of-life enhancements include enhanced Weights & Biases integration and automated batch size setting.

The text discusses the upcoming availability of FLUX Tools via Together APIs, which will provide greater control over image generation using Canny and Depth models. Users can get started with FLUX by trying out the models in their playground or connecting via API. Free access to FLUX.1 [schnell] is available until the end of the year through the FLUX.1-schnell-Free endpoint. The RedPajama-V2 Dataset, designed as a foundation for creating high-quality datasets, should not be used out of the box and requires filtering based on specific applications.

Together AI and Hypertec Cloud are co-building one of the world's largest optimized GPU clusters, featuring 36,000 NVIDIA GB200 GPUs. This partnership combines Together AI's high-performance GPU Clusters and deep AI research expertise with Hypertec Cloud's infrastructure compute and data center capabilities to deliver next generation infrastructure for accelerating training, fine-tuning, and inference of large generative AI models. The integrated hardware and software stack enhances performance, scalability, and cost efficiency by optimizing GPU usage, enabling efficient scaling for AI workloads while reducing training time and operational costs.