August 2024 Summaries

This article discusses best practices for fine-tuning large language models (LLMs) with a focus on model selection and curation. It highlights the importance of choosing the right base model, training dataset, and task complexity to achieve optimal performance. The article compares proprietary OpenAI models with open-source models, noting that open-source models have strong performance with less data but may require more data to reach higher performance ceilings. It also discusses factors such as ease of use, cost, control, and hyperparameter tuning when fine-tuning models. The article emphasizes the importance of using sensible defaults most of the time, keeping it simple, and having a robust evaluation suite to measure performance. Ultimately, it suggests that open-source models can provide significant cost savings and flexibility in deployment options, making them a viable alternative to proprietary models.

Fine-tuning large language models (LLMs) requires high-quality training data to achieve optimal performance. The ideal training data should match the real-world input domain as closely as possible, with adequate coverage of various tasks and inputs. Collecting optimal training data involves leveraging pre-existing production application logs, using OpenPipe's SDK for automatic logging, or pulling relevant data from databases. Human-generated data can be effective when leveraged by domain-experts, but manual creation by non-specialists is not recommended due to quality issues. Synthetic data on the input side can lead to model quality issues, whereas synthetic data on the output side can be effective with human review and patching. Agentic approaches, such as OpenPipe's Mixture-of-Agents technology, can generate high-quality outputs superior to SOTA models. The amount of training data required varies depending on the task complexity and base model size, but a general rule is that larger base models require less training data for improved performance.