February 2024 Summaries

Vespa, a leading vector database, is praised for its exceptional embedding flexibility, as highlighted in the recent GigaOm Sonar Report. Users can integrate embeddings in four ways: independently outside Vespa, using their own models within Vespa, through Vespa's model hub, or with custom code, offering a high degree of adaptability. This flexibility allows for the creation and indexing of multiple embeddings per document, enabling experimentation with various models without altering the application's core. The database supports simultaneous use of different embedding methods for different fields and facilitates easy transitions between them. Marqo, a vector search expert, chose Vespa over competitors like Milvus and OpenSearch, demonstrating its capabilities in the field. Vespa also offers resources for newcomers, including a free trial with $300 in credits, encouraging users to explore its potential.

The Vespa Newsletter for February 2024 highlights several advancements and recognitions for Vespa.ai, including its recognition as a leader in GigaOm's report on vector databases due to its strengths in model versatility, search flexibility, and real-time capabilities. The update introduces new features such as a native ColBERT embedder, enhancements to YQL with the addition of an IN operator, and support for fuzzy and regexp matching in streaming search. It also discusses performance improvements for fuzzy matching and the use of parameter substitution to simplify hybrid queries. Additionally, Vespa is sponsoring hackathons at Stanford and Berkeley, and the newsletter invites readers to explore new blog posts and try Vespa Cloud for free.

The blog post discusses the evolution and challenges of using vector databases for search, emphasizing that while vectors are often used to enhance data retrieval, reliance solely on vector-based search is insufficient. It highlights the superiority of hybrid search approaches that combine lexical and vector similarity for better results, particularly in text search, and underscores the importance of combining various signals, including structured data and text matching, into a comprehensive ranking score. The post argues against a divided approach to handling search tasks due to scalability issues and advocates for a unified platform that integrates all necessary components to efficiently manage and deploy search applications, which is the rationale behind the design of the Vespa platform.

The Vespa team announced the availability of a native implementation of the ColBERT embedder, a sophisticated semantic search tool that leverages token-level vector representations for improved search explainability and ranking quality. Unlike typical text embedding models that compress information into a single vector, ColBERT uses contextualized token vectors, enabling more precise similarity comparisons and transparency in scoring with its MaxSim function. The new Vespa implementation features a novel asymmetric compression technique that significantly reduces storage requirements without sacrificing ranking accuracy. This advancement enhances the developer experience by reducing the vector storage footprint by up to 32 times. ColBERT's architecture, which separates query and document processing, supports pre-computation and fine-tuning with fewer labeled examples. The Vespa platform integrates ColBERT with ease, allowing users to deploy it alongside other ranking models and highlighting its compatibility with various applications, including those requiring long-context handling through chunking. The blog post also provides a comprehensive FAQ section to address common concerns about using ColBERT within Vespa, emphasizing its advantages in terms of interpretability, efficiency, and integration with existing Vespa features.

Vespa has been recognized as a leader in the GigaOm Sonar report for vector databases, highlighting its strengths in embedding flexibility, rapid updates, and hybrid search capabilities with neural network rankings. These features are critical for maximizing the return on investment for vector similarity search engines, as they allow applications to manage and evolve embedding models while serving queries and handling normal operations. Vespa's platform supports real-time changes to vectors, text, and metadata independently, enabling high-volume updates without rewriting vectors. It also excels in combining vector similarity with text matching and metadata signals to achieve high-quality relevance beyond simple vector similarity and bm25 scores. By allowing machine-learned models to operate locally on data nodes, Vespa ensures scalable performance, handling hundreds of thousands of requests per second, which has been validated by its engineering-first approach and long-standing track record in AI-driven applications.

In this blog post, Andreas Eriksen, a senior Vespa engineer, explores the integration of OpenAI's text-embedding-3 embeddings with Vespa, focusing on the Matryoshka Representation Learning (MRL) technique. MRL allows embeddings to be shortened without losing their concept-representing properties, enabling smaller embedding sizes, faster searches, and efficient storage. The post discusses using phased ranking to re-rank top results with full embeddings for accuracy comparable to full-size embeddings. An information retrieval benchmark evaluates the quality of results with various embedding sizes and retrieval strategies. The blog also demonstrates the creation of Vespa schemas, rank profiles, and the deployment of applications to Vespa Cloud, with specific focus on embedding flexibility and query optimization. The experiment highlights the trade-off between performance and accuracy, revealing that even shortened embeddings yield good results with significant memory savings and reduced latency.