Virtuoso-Lite & Virtuoso-Medium-v2: Distilling DeepSeek-V3 into 10B & 32B Small Language Models (SLMs)

Arcee AI has introduced two new small language models (SLMs), Virtuoso-Lite and Virtuoso-Medium-v2, as part of their ongoing commitment to transforming advanced research into practical AI tools. Virtuoso-Lite is a 10B-parameter model derived from TII's Falcon architecture, featuring innovations such as tokenizer work and distillation with fp8 DeepSeek-V3 to maintain performance while reducing parameter count. Virtuoso-Medium-v2, a 32B distillation of DeepSeek-V3, surpasses previous models like Arcee-Nova 72B in benchmarks, demonstrating the efficacy of their logit-level distillation pipeline. Both models are released under the Apache-2.0 license, allowing for wide integration into various projects, and they are available on platforms like Hugging Face and Arcee's inference platform for easy access. Arcee AI plans to continue their efforts with upcoming R1 distillations, aiming to provide even more powerful models in the future.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.