September 2022 Summaries

Embeddings are crucial for modern semantic search and neural ranking, and Vespa.ai has simplified their creation and use by integrating an embedding feature directly into its platform. Previously, users needed to create embeddings on the client side or develop Java components, but with Vespa 8.54.61 or higher, embedding creation is streamlined through a simple addition to the services.xml file using a BERT-style model and vocabulary, such as the recommended model from Hugging Face. This allows for automatic conversion of text queries into embeddings, facilitating efficient search and evaluation of machine-learned models. Vespa.ai also provides a sample application for semantic search to help users get started, with additional resources available for those interested in exploring the integration of pretrained transformer models and vector search efficiency.

The emergence of vector databases poses questions about their potential to replace traditional search engines, yet these new systems generally lack features such as phrase search and dynamic summaries that are critical for comprehensive search engine implementations. Traditional search engines like Apache Lucene and Vespa have decades of development, offering features like accelerated dynamic pruning algorithms and real-time signal integration, which are challenging to replicate with dense vector calculations alone. While hybrid search models combining sparse and dense vectors prove more effective, integrating dense vector capabilities into traditional engine architectures can lead to increased latency and costs. Vespa, an open-source big data serving engine, presents a viable alternative by implementing mutable HNSW graphs and facilitating efficient hybrid search without the drawbacks of immutable data structures. Vespa's architecture ensures low latency and eliminates the need for full re-indexing when updating data, offering a scalable solution with proven ranking results on extensive open relevancy datasets.

In the September 2022 update of the Vespa Newsletter, several key enhancements were highlighted, including the introduction of rank-phase statistics, schema feeding flexibility, and new beta applications for query building and trace visualization. Rank-phase statistics provide insights into query performance on a per-document level, aiding in real-time ranking performance tracking. Schema feeding flexibility, introduced in Vespa 8.20, allows document feeds to contain unknown fields, simplifying schema optimization and evolution. The beta applications, available on GitHub, are designed to facilitate query experimentation and help identify query latency issues. Additionally, rank trace profiling, available since Vespa 8.48, provides detailed information on time distribution across rank features, and a feeding bandwidth test feature was introduced in Vespa 8.35 to differentiate between application latency and client-server bandwidth issues. The update also mentions Vespa's capability to integrate custom Java code and the usefulness of query tracing and debugging in developing advanced use cases.

The tutorial demonstrates building a text search application using Vespa, focusing on evaluating and comparing two ranking functions, NativeRank and BM25, through various metrics such as recall, reciprocal rank, and normalized discounted cumulative gain. It involves setting up a Vespa application package for passage ranking experiments, deploying it in a Docker container, and feeding it with a sample of the passage ranking dataset. The process includes querying the application using both a QueryModel and Vespa Query Language (YQL), and assessing the models' performance by obtaining point estimates and computing uncertainty estimates using bootstrap sampling. The results are visualized using plots to highlight the differences in ranking functions, enhancing the understanding of their impact on application performance. The tutorial also addresses the significance of measuring uncertainty in evaluation metrics to better interpret the effects of changes in ranking functions.