July 2023 Summaries

The blog post explores the implementation of multilingual embedding models from the E5 family within Vespa, an open-source engine designed for large-scale data applications. It details how these models transform textual data into a unified vector space, facilitating multilingual information retrieval by allowing queries and documents from different languages to be compared seamlessly. The post highlights the trade-offs between model size, accuracy, and computational costs, noting that larger models increase storage and computational demands linearly, but not necessarily accuracy. It discusses the benefits of using Vespa's vector search and embedding inference capabilities, which enable developers to integrate multilingual semantic search into applications without complex infrastructure management. The evaluation of the small E5 variant on various datasets, including BEIR and MIRACL, shows that while dense embedding models can outperform traditional methods like BM25 in some contexts, they may underperform in low-resource language scenarios due to a lack of pre-training. The article concludes with a look ahead at optimizing embedding inference costs and improving retrieval effectiveness through precision adjustments.

Vespa's July 2023 newsletter highlights several advancements, including the introduction of Vector Streaming Search, which enables efficient and cost-effective vector searches in personal data applications. The platform now supports embedding models from Huggingface, allowing developers to import these models in ONNX format for accelerated inference. Additionally, GPU acceleration has been introduced for embedding model inferences, enhancing the processing speed and scalability of large text data. Vespa Cloud users have access to the E5 family of text embeddings, known for their adaptability across various tasks. The newsletter also details the support for a dotproduct distance metric for Maximum Inner Product Search (MIPS) and improvements in searching capabilities, such as emoji indexing and multivalue field sorting. The update includes various performance optimizations, such as reduced static memory usage for multivalue attributes, and highlights recent blog posts and events, encouraging users to explore Vespa Cloud and the latest releases.

Leveraging frozen embeddings within the Vespa search application using SentenceTransformers offers a streamlined approach to managing the complexity of hybrid search systems, particularly in dynamic environments like e-commerce where search patterns frequently change. By freezing document vector representations and updating only query representations, this method reduces the need for frequent recalculation of embeddings when models are retrained, thus easing the maintenance burden. The article details the implementation of a bi-encoder model with asymmetric dense layers to achieve frozen embeddings, utilizing the sentence-transformers library for training, and integrating these models into Vespa through ONNX format exportation and custom embedding components. This approach not only facilitates efficient memory usage by sharing transformer weights between document and query models but also offers a plug-and-play training procedure for embedding generation, ultimately enhancing the manageability and scalability of Vespa applications.

Vespa has introduced a new feature to address Maximum Inner Product Search (MIPS) challenges by transforming them into Nearest Neighbor Search (NNS) problems using a dotproduct distance metric. MIPS is particularly useful in scenarios like recommender systems and machine learning applications where vectors representing items and user preferences are scored using the dot product. Vespa's approach involves adding an extra dimension to vectors, allowing for dynamic indexing without prior knowledge of vector norms, and adapting incrementally as data is added. This transformation enables Vespa to use the negative dot product as a distance metric, facilitating efficient search and retrieval in high-dimensional spaces. Experiments conducted with the Wikipedia dataset demonstrated that ordering documents by descending embedding vector norm can enhance recall, though satisfactory results are also achieved with random order, mimicking real-world data input. Users can further optimize recall by adjusting HNSW index settings and the exploreAdditionalHits parameter.