March 2025 Summaries

Vespa Voice is a newly launched podcast aimed at exploring the future of AI-driven search infrastructure, featuring insights from leaders and innovators in the field. The first episode delves into the rapidly growing vector database landscape, with Whit Walters from GigaOM discussing Vespa's recognition as a leader and fast mover, highlighting its scalability, cost predictability, and performance advantages over traditional databases. Vespa, a platform for developing real-time AI-driven applications, excels in search, recommendation, personalization, and retrieval-augmented generation, capable of handling vast data sets with low latency. Future episodes promise to cover cutting-edge applications, enterprise AI trends, and real-world cases of Vespa's implementation, offering valuable insights for those interested in vector databases and AI infrastructure.

In an effort to enhance the efficiency of lexical search, Vespa has introduced several tunable query-time optimizations that reportedly triple the performance of natural language text search, according to the blog post by Vespa engineers. These optimizations involve reducing the precision required for very common words, filtering out such words automatically, and minimizing the number of internal result candidates to lower ranking costs, all while maintaining a marginal expected loss of query result quality. These changes are available in Vespa version 8.503.27 and leverage a hybrid search model combining lexical and vector search techniques to balance specificity and semantics. The post details how these optimizations reduce the reliance on large posting lists and improve query performance through parameters like filter-threshold, stopword-limit, and adjust-target, which can be adjusted in a rank profile or overridden per query. Experiments with datasets from the BEIR benchmark show that these changes optimize query performance without significantly affecting quality, especially when used in conjunction with Vespa's weakAnd query operator and a modified MMAP madvise setting for better I/O subsystem utilization. This advancement promises reduced hardware costs, lower query latencies, and improved stability, particularly in environments with limited physical memory.

ModernBERT, a new iteration of the BERT model announced by Answer.AI and LightOn.ai, integrates the latest advancements in architecture, training techniques, and dataset curation, resulting in a base model with significantly improved efficiency and performance. ModernBERT, which retains the core functionality of BERT, is designed as a general-purpose model for tasks like feature extraction, text classification, and retrieval, and it supports a longer context length of up to 8192 tokens. It is particularly effective in code retrieval and long context retrieval tasks due to its code-friendly tokenizer and enhanced context capabilities. Vespa, an open big data serving engine, can incorporate ModernBERT models, providing configurations for various embeddings available on the Huggingface Hub, including those developed by Nomic AI, LightOn AI, and Alibaba. These configurations utilize metrics such as angular distance and support Matryoshka, enabling a balance between accuracy and efficiency. Vespa encourages experimentation with ModernBERT through its platform, offering a free trial to explore its potential in transforming retrieval pipelines.

Emerging video search use cases in enterprise environments have driven interest in advanced retrieval systems that extend beyond simple transcript searches, prompting exploration into more sophisticated solutions like Vespa and TwelveLabs. These systems address complex requirements such as searching based on visual content and context within videos, which traditional methods like audio-to-text conversion might not fully capture. TwelveLabs offers a multi-modal embedding model capable of capturing visual expressions, body language, spoken words, and overall video context, while Vespa provides a robust platform for scalable video storage and search, utilizing billion-scale vector search and hybrid search capabilities that combine lexical and semantic approaches. Vespa's advanced ranking capabilities, including a multi-phase ranking approach, allow efficient retrieval and ranking of relevant video clips amidst extensive video collections. The text details the implementation of video search using TwelveLabs' embedding models and Vespa's distributed architecture, demonstrating the integration of these technologies to perform complex video searches with enriched metadata and multi-vector representations, alongside a practical example using sample videos.