DenseOn with the LateOn: Open State-of-the-Art Single and Multi-Vector Models

LateOn and DenseOn are two newly released open retrieval models that exceed the performance of existing state-of-the-art models on the BEIR benchmark. Developed with the ModernBERT backbone, these models boast a parameter count of 149 million, strategically balancing size with complexity to handle real-world queries efficiently. LateOn, focusing on multi-vector ColBERT methods, achieved an NDCG@10 score of 57.22, while DenseOn, a single-vector dense retriever, scored 56.20. Both models demonstrate robust generalization capability, confirmed by decontamination experiments that strip overlapping training data from evaluation corpora, thereby ensuring improvements result from genuine generalization rather than data leakage. The models were developed through a two-stage training pipeline involving large-scale unsupervised contrastive pre-training followed by supervised fine-tuning on a curated dataset with mined hard negatives. Released under Apache 2.0, these models and their training data are accessible for community use, fostering further research and innovation in open retrieval model development.