Modernize your retrieval pipeline with ModernBERT and Vespa

ModernBERT, a new iteration of the BERT model announced by Answer.AI and LightOn.ai, integrates the latest advancements in architecture, training techniques, and dataset curation, resulting in a base model with significantly improved efficiency and performance. ModernBERT, which retains the core functionality of BERT, is designed as a general-purpose model for tasks like feature extraction, text classification, and retrieval, and it supports a longer context length of up to 8192 tokens. It is particularly effective in code retrieval and long context retrieval tasks due to its code-friendly tokenizer and enhanced context capabilities. Vespa, an open big data serving engine, can incorporate ModernBERT models, providing configurations for various embeddings available on the Huggingface Hub, including those developed by Nomic AI, LightOn AI, and Alibaba. These configurations utilize metrics such as angular distance and support Matryoshka, enabling a balance between accuracy and efficiency. Vespa encourages experimentation with ModernBERT through its platform, offering a free trial to explore its potential in transforming retrieval pipelines.