Fine-Tuning Phi-3 & Gemma 2: The Budget Path to GPT-4 Performance at a Fraction of the Cost

Phi-3 and Gemma 2 are AI models developed by Microsoft and Google, respectively, designed for cost-effective fine-tuning by enterprises without compromising quality. A study demonstrated that Microsoft's Phi-3-mini, a 3.8 billion parameter model, significantly outperformed GPT-4o on six out of seven financial NLP benchmarks, achieving a 96% accuracy in financial headline classification compared to GPT-4o's 80%, while being approximately 29 times cheaper in inference costs. Google's Gemma 2, a 9 billion parameter model, approaches early GPT-4 performance in human preference evaluations and is optimized for practical deployment even on consumer hardware like the RTX 4090. The guide provides insights on fine-tuning these models for under $100 in compute costs, emphasizing their suitability for specific tasks, with Phi-3 excelling in analytical workloads and Gemma 2 in conversation-heavy applications. The document argues that these specialized models can outperform general-purpose large language models (LLMs) like GPT-4 on domain-specific tasks, offering significant cost savings for enterprises. Despite their advantages, these models are not suitable for tasks requiring broad general knowledge or highly unpredictable queries, where general-purpose LLMs maintain superiority.