Announcing Maximum Inner Product Search

Vespa has introduced a new feature to address Maximum Inner Product Search (MIPS) challenges by transforming them into Nearest Neighbor Search (NNS) problems using a dotproduct distance metric. MIPS is particularly useful in scenarios like recommender systems and machine learning applications where vectors representing items and user preferences are scored using the dot product. Vespa's approach involves adding an extra dimension to vectors, allowing for dynamic indexing without prior knowledge of vector norms, and adapting incrementally as data is added. This transformation enables Vespa to use the negative dot product as a distance metric, facilitating efficient search and retrieval in high-dimensional spaces. Experiments conducted with the Wikipedia dataset demonstrated that ordering documents by descending embedding vector norm can enhance recall, though satisfactory results are also achieved with random order, mimicking real-world data input. Users can further optimize recall by adjusting HNSW index settings and the exploreAdditionalHits parameter.