Mixtral Curious? Comparing Mistral 7B and Mixtral for fine-tuning

Mixtral and Mistral 7B are two popular base models released by Mistral AI, with Mixtral being the stronger but more expensive model. The author of the article has fine-tuned hundreds of Mistral 7B models and dozens of Mixtrals, allowing them to compare their performance in a series of evaluations. These evaluations showed that Mixtral wins around 60% of its head-to-head comparisons with Mistral, although when ties are included, Mixtral's win rate is only 53%. The article concludes that while both models are strong, Mixtral is the better choice if high quality is non-negotiable and cost isn't a concern. However, smaller models like Phi-2 or Gemma 2B might be more suitable for certain tasks where lower costs and latency are important.