January 2026 Summaries

RTEB (Retrieval Embedding Benchmark) is a new benchmark developed by MTEB to evaluate retrieval capabilities in real-world industry scenarios, addressing issues like model overfitting on public datasets. Focusing on domains such as legal, finance, healthcare, and code, RTEB utilizes a hybrid evaluation strategy with both open and private datasets to measure true generalization capabilities. The Octen series models, particularly the Octen-Embedding-8B, achieved first place on the RTEB leaderboard by leveraging systematic technical optimizations, such as domain-specific synthetic data, data optimization strategies like Hard Negative sampling, and training efficiency improvements through methods like LoRA fine-tuning. Despite challenges regarding evaluation fairness due to unequal access to private datasets, Octen models demonstrated their generalization capabilities and maintained high performance across both public and private datasets. The series has been open-sourced on the Hugging Face platform, promoting technological progress and community contribution in the field of retrieval and embedding technologies.