Phi-4 Finetuning + Bug Fixes by Unsloth

Microsoft's new Phi-4 model, integrated into Unsloth, rivals OpenAI's GPT-4o-mini in performance and has recently undergone significant improvements to enhance its accuracy. These enhancements include resolving four key bugs related to tokenization, fine-tuning, and chat template issues, while also converting the model to Llama's architecture for improved accuracy and ease of use. The fine-tuning process is now twice as fast, uses 70% less memory, and supports context lengths over 128K, which is substantially longer than previous models. The updates also introduce dynamic 4-bit quantization, which boosts accuracy without a significant increase in VRAM usage. These changes have been well-received, with feedback from Reddit users indicating improved model performance on the Hugging Face OpenLLM Leaderboard. Users can experiment with fine-tuning Phi-4 using Unsloth's Colab Notebook, which is compatible with Google's free Tesla T4 16GB GPU, providing an accessible platform for further customization and testing.