December 2024 Summaries

EuroLLM-9B is a 9-billion-parameter open language model released in December 2024 by a consortium of European research and industry organizations to improve multilingual AI coverage, particularly for Europe’s 24 official EU languages alongside 11 additional international languages. Developed using EuroHPC computing resources, it follows the earlier EuroLLM-1.7B model and is available in pretrained and instruction-tuned versions on Hugging Face. The model uses a 128,000-piece tokenizer designed for European languages and was trained on roughly 4 trillion tokens using 400 Nvidia H100 GPUs on the MareNostrum5 supercomputer, with training progressing from broad web and curated multilingual data to increasingly high-quality data during annealing stages. Benchmark results presented by the developers indicate that EuroLLM-9B performs strongly on multilingual general-language, translation, and English-language tests, ranking ahead of other European-developed models of similar size and competitively against models such as Gemma-2-9B and Mistral-7B. Its instruction-tuned version is intended for multi-turn conversations and translation across official EU languages, while the project plans larger future models and acknowledges the challenge of obtaining sufficiently high-quality data for additional regional and minority European languages.