March 2026 Summaries

Vespa has introduced a new feature in its embedders to automatically handle large ONNX models that use external data files, addressing a long-standing limitation due to the 2 GB protobuf format cap. This enhancement allows Vespa embedders to download external data files referenced by ONNX models, enabling the integration of models that were previously too large to use directly within Vespa's built-in systems. The update, available from Vespa version 8.544, supports models used in embedders such as hugging-face-embedder, colbert-embedder, and splade-embedder, with the ability to use private models by propagating authentication tokens when downloading external data. However, embedders are the only components that currently support models with external data, and there are still constraints regarding model usage in ranking expressions, requiring models to be self-contained and under 2 GB. Additionally, external data files must be located in the same directory or a subdirectory relative to the model URL, and further support for ONNX models with external data in ranking expressions can be requested through issue tracking.

Metal AI, a platform designed to provide institutional intelligence for private equity firms, utilizes Vespa Cloud to overcome the limitations of traditional document-centric retrieval systems by focusing on entity and relationship-driven retrieval. This approach enables Metal to transform fragmented deal data into a comprehensive system of record, enhancing investment decision-making by drawing on historical and live data. Vespa's multi-entity modeling, advanced ranking and filtering capabilities allow Metal to integrate business logic with semantic similarity, facilitating the creation of sophisticated workflows like Due Diligence Questionnaire automation. By leveraging Vespa's flexibility and operational simplicity, Metal can rapidly adapt its retrieval strategies and scale without the need for extensive re-architecture, ultimately supporting an AI agent-driven approach that efficiently manages complex queries and evolving data models.

Asymmetric retrieval, a new approach supported by Vespa and Voyage AI's voyage-4 family, offers a cost-effective method for handling high-volume queries by embedding documents with a robust model while using a smaller, local model for queries. This method leverages the fact that document embedding is a one-time cost, allowing for the use of more resource-intensive models without latency concerns, whereas query embedding is continuous and cost-prohibitive at scale. By embedding documents with the highly accurate voyage-4-large model and queries with the lightweight voyage-4-nano model, companies can significantly reduce costs, as demonstrated by potential savings of $15,000 monthly at 10,000 queries per second. This strategy maintains retrieval quality through a shared vector space and phased ranking system, which combines the speed of binary search with the accuracy of full-precision reranking, making it feasible for large-scale production environments. Vespa's architecture, which separates query embedding from content storage, supports independent scaling and eliminates dependency on external APIs, ensuring stability even during API outages or traffic spikes.

Retrieval-Augmented Generation (RAG) on Vespa Cloud offers an efficient solution for grounding large language model (LLM) responses in real, trusted data sources by bridging the gap between LLMs' fixed knowledge and proprietary datasets. The key challenge in RAG is optimizing the LLM's context window to ensure high-quality, relevant information retrieval, which Vespa addresses by combining semantic vector retrieval with lexical BM25 scoring and advanced ranking models. Vespa Cloud's out-of-the-box RAG Blueprint facilitates the rapid deployment of a high-quality retrieval stack, enabling users to build end-to-end RAG applications in about 15 minutes. This involves setting up data ingestion pipelines, query processing flows, and a lightweight chat UI that allows users to interact with their data. Vespa's hybrid retrieval approach, which integrates vector similarity with BM25 text matching, is further enhanced by various query profiles, offering flexibility and precision in search results. With Vespa Cloud, users gain access to scalable, reliable infrastructure equipped with auto-scaling and observability features, making it suitable for both small-scale experiments and large-scale deployments.