May 2023 Summaries

The May 2023 Vespa Newsletter highlights significant updates, including the introduction of multi-vector HNSW indexing, which addresses the limitation of representing data items with a single vector by allowing indexing of multiple vectors per document. Additionally, Vespa now supports declarative global re-ranking, enabling GPU-accelerated inference to optimize search ranking without writing code. Integration with LangChain allows Vespa to act as a data source for language model-powered applications, and several optimizations have been introduced, such as improved bfloat16 tensor operations and new features in the Vespa Cloud Dev Console. The newsletter also mentions the availability of Vespa on Docker Hub without rate limits and outlines the benefits of using the Vespa CLI for data management. Subscribers are encouraged to keep up with updates via the Vespa Blog RSS feed and are reminded of Vespa's daily production releases for enhanced security and performance.

Vespa has introduced a high-performance feed client within the Vespa CLI, designed to efficiently handle document feeding and streamline operations for Vespa applications. This new client aims to replace the older vespa-feed-client, which, despite its efficient performance, relies on a cumbersome Java runtime and lacks user-friendly features like automatic authentication configuration. The new feed client is integrated directly into Vespa CLI, allowing users to manage both self-hosted installations and Vespa Cloud applications without switching tools. While the performance of the new client surpasses the old client for small and medium document sizes, it still falls short for larger documents, but improvements are ongoing. The vespa-feed-client will remain supported, offering both a Java library and a command-line interface, although users are encouraged to adopt the new client if they don't need the Java library. Vespa CLI version 8.164 includes this update, with installation and upgrade options available through Homebrew or GitHub, and additional support provided through Vespa's community channels and documentation.

Vespa's latest update introduces support for declarative global re-ranking, allowing for the streamlined deployment of multi-phase ranking pipelines at scale without the need for extensive coding or complex infrastructure management. This enhancement is particularly beneficial in the context of integrating Large Language Models (LLMs) with text retrieval systems, where accurate retrieval and ranking are crucial to prevent LLMs from generating inaccurate responses due to irrelevant context. Vespa leverages multi-vector and cross-encoder models, which perform better in zero-shot settings without in-domain fine-tuning, to improve ranking accuracy. The update enables the use of phased ranking, where each stage of the retrieval process filters out less relevant documents, culminating in a global re-ranking phase that employs ONNX models for inference. This process benefits from GPU acceleration, reducing costs and enhancing performance through Vespa Cloud's autoscaling capabilities. The new feature is available from Vespa version 8.164 and aims to improve the robustness and efficiency of search applications by employing state-of-the-art cross-encoders, as demonstrated on the BEIR benchmark.

Vespa has introduced support for Langchain, simplifying the use of large language models (LLMs) in applications by incorporating a Vespa retriever into Langchain's framework. This integration is aimed at enhancing the functionality for developers working with LLMs, making it easier to experiment and deploy these models. The collaboration between Vespa and Langchain is acknowledged, with appreciation extended to Langchain creator Harrison Chase for his insights regarding Vespa's integration. Further details and documentation on this development can be found on the Langchain website.