Mistral Small 3 Avoids Synthetic Data—Why That Matters

Mistral AI has launched Mistral Small 3, a 22 billion parameter model featuring 32k context, which notably does not use synthetic data in its training, making it suitable for creative applications where nuanced language is crucial. Synthetic data, while beneficial in structured domains like programming due to its ability to mimic statistical properties of real data, poses challenges in capturing the subtleties and complexities of real-world language, potentially leading to models that struggle with authenticity and creative expression. As synthetic data can perpetuate biases and lacks the ability to fully replicate real-world scenarios, its use is more beneficial in predictable domains, whereas models without synthetic data, like Mistral Small 3, are advantageous in spontaneous and creative contexts. The model is designed for efficient deployment, running at full weights on an A40 GPU and is compatible with various quantization levels, making it accessible for users with different hardware specs, and offering a potential alternative to larger models that rely on synthetic data.