Fine-tuning small open-source LLMs to outperform large closed-source models by 60% on specialized tasks

Baseten is demonstrating how small open-source models, when paired with rigorous evaluation and task-specific optimization, can outperform larger proprietary models in complex real-world applications, such as healthcare scribing. By leveraging the concept of compute-optimal training, Baseten emphasizes the importance of balanced parameter-to-token ratios and the effectiveness of smaller, task-optimized models that can deliver 60% better accuracy, lower inference costs, and faster processing times compared to larger models. Baseten's approach involves building a programmatic, domain-aligned evaluation system that breaks tasks into granular checks and integrates these into training and deployment pipelines. This evaluation-first methodology, which includes the use of mechanistic interpretability techniques, not only enhances model performance but also ensures transparency and reliability. In a healthcare use case, Baseten's approach to fine-tuning a 27B parameter model resulted in surpassing the performance of larger models such as Claude Sonnet 4, achieving significantly lower latency and cost while maintaining high accuracy and reliability. Their methodology includes a sophisticated evaluation harness that aligns with expert clinical judgment and supports continual reinforcement learning, thus providing a foundation for sustained improvement and cost efficiency in domain-specific applications.