August 2023 Summaries

The Vespa Newsletter from August 2023 highlights recent updates and features in the Vespa AI platform, including the introduction of a multilingual sample app that leverages Vespa’s indexing language and embedding support for enhanced search relevance. The update also discusses improvements in approximate nearest neighbor queries with the targetHits feature, optimizations in the Vespa command-line client for easier deployment, and the release of new PyVespa versions that enhance interoperability with Vespa Cloud. Additional enhancements include the support for converting ISO-8601 date strings to epoch time, using onnxruntime 1.15.1, and optimizing query performance through selective group upgrades in high-query applications. The newsletter also mentions recent blog posts covering topics like BGE embedding models and accelerating transformer-based embedding retrieval, inviting users to try Vespa on Vespa Cloud or through the latest release.

During a summer internship at Vespa, two interns embarked on a project to enhance semantic search relevance by developing a system that automates the creation of training data for text embedders using large language models (LLMs) like ChatGPT. The aim was to generate both queries and query relevance judgments (qrels) automatically, thereby reducing the manual labor traditionally involved in this process. They utilized techniques such as few-shot prompting to improve output quality and experimented with various datasets, achieving the most success with the NFCorpus dataset. Despite challenges in applying the system to other datasets, the interns identified potential improvements, including using different models, refining training parameters, and implementing frozen embeddings to overcome bottlenecks in large datasets. Additionally, they engaged in side projects such as developing a sample app for creating embeddings and enhancing the Pyvespa Python API, which bolstered their understanding of Vespa's capabilities and contributed to their professional growth. The internship offered valuable insights into information retrieval, machine learning, and open-source project contributions, with strong support from the Vespa team.

In the blog post, Jo Kristian Bergum explores the integration of BGE (BAAI General Embedding) models into Vespa, highlighting their efficiency on the Massive Text Embedding Benchmark (MTEB) and their performance on the BEIR trec-covid dataset. The post details how Vespa's support for storing vectors using bfloat16 precision reduces memory usage by 50% with minimal impact on retrieval quality. It compares three BGE model variants, emphasizing the balance between accuracy and computational costs, and shows that quantization can enhance CPU inference efficiency. Additionally, the post discusses exporting BGE models to ONNX format for optimized inference and using Vespa's native embedding support for seamless deployment across different environments, eliminating the need for separate systems to manage embedding inference and nearest neighbor search.

Vespa Blog's post, authored by Chief Scientist Jo Kristian Bergum, explores accelerating transformer-based embedding retrieval by utilizing the Vespa platform, focusing on embedding inference and retrieval with nearest neighbor search. The blog emphasizes the role of text embedding models, particularly encoder-only transformer models like BERT, in mapping text to vector spaces for multilingual retrieval. It discusses the complexity of inference, the importance of sequence length, and the trade-offs between model size and accuracy. Through experiments using Vespa on a laptop, it demonstrates performance improvements with techniques like post-training quantization and approximate nearest neighbor search, leading to enhanced throughput and reduced latency without significantly sacrificing retrieval quality. The article highlights Vespa's ability to streamline embedding inference and retrieval processes, offering a flexible, efficient solution for deploying embedding models across various environments without the need for separate infrastructure.