October 2022 Summaries

The Vespa Newsletter for October 2022 announces several updates and improvements to the Vespa.ai platform, highlighting its enhanced capabilities and flexibility. Users can now create vector embeddings without custom Java code, leveraging BERT-style models for semantic similarity in document retrieval. Vespa Cloud simplifies this process further by providing pre-integrated machine learning models. The platform introduces improved query performance through bitvector optimization for frequent terms and extended paging support for all attribute types, excluding tensor with fast-rank and predicate. Additionally, multi-platform support now includes ARM64, and query result highlighting has been expanded to arrays of strings. The transition of Vespa scripts to Go from Perl aims to broaden compatibility across client platforms.

In the second part of a blog series on building a billion-scale vector search using Vespa, the focus is on balancing cost and performance for large-scale vector search solutions, particularly with approximate nearest neighbor approaches. The blog discusses the challenges of handling vast amounts of unstructured data and the need for cost-efficient query processing, using the LAION-5B dataset as a case study. This dataset provides large-scale vector representations useful for training models like StableDiffusion and is leveraged to build a searchable multi-modal index. The blog outlines a hybrid search method combining sparse and dense vector representations, using techniques such as PCA for dimension reduction to optimize memory usage and computational efficiency. It emphasizes a phased retrieval and ranking approach, where initial coarse-level searches are conducted on reduced vector spaces to limit data movement and computational load, followed by more refined searches. The piece also highlights the advantages of using a tiered compute approach, moving some vector similarity calculations to stateless clusters for faster auto-scaling with changes in query volume, thus reducing costs in cloud environments. This methodology supports dynamic scaling and efficient resource use, crucial for handling fluctuating query volumes without excessive overhead.

Vespa.ai, an open-source platform since 2017, has expanded its container image support to include both x86_64 and ARM64 architectures, enhancing its versatility across different hardware platforms. The Vespa team has extensively tested ARM64 to ensure optimal performance, allowing application owners to benchmark and select the most cost-effective and efficient CPU architecture for their needs. This multi-platform support facilitates easier development and deployment workflows, particularly benefiting developers using ARM-based devices like Apple's M1 MacBooks, as they can seamlessly work with the vespaengine/vespa image. The new capabilities also extend to Vespa Cloud, providing users with more options for resource and vendor optimization.

Advancements in self-supervised deep learning have significantly enhanced vector search capabilities, enabling the processing of billions of vectors in milliseconds, which is crucial for handling unstructured data across various modalities such as text, audio, and images. Despite the challenges associated with scaling vector search for large datasets, particularly for organizations with extensive data but lower query traffic, hybrid algorithms using solid-state disks and in-memory data structures offer a cost-efficient solution. Vespa, an open-source big data serving engine, facilitates large-scale vector search and recommendation use cases with its versatile schema model and real-time implementation of the HNSW algorithm for approximate nearest neighbor search. This approach provides a framework for organizations to efficiently manage and scale vector search operations, balancing the need for low latency and high accuracy with economic considerations. Vespa's architecture supports the independent scaling of content and compute resources, enabling on-demand and elastic auto-scaling, which is beneficial for applications requiring high throughput and rapid query responses.

Vespa Cloud now offers pre-trained models that can automatically convert text to embeddings, simplifying the integration of machine learning models in applications by eliminating the need to upload models individually. Users can select desired models from Vespa's model hub and incorporate them into their applications by referencing a model-id. This update enhances the ease of deploying pre-trained models for various tasks, allowing for seamless integration within Vespa's configurations or custom setups. Over time, Vespa Cloud plans to expand its exclusive model offerings to ensure high performance and quality, thereby enabling more teams to effectively utilize AI in production environments.