Introducing cascading retrieval: Unifying dense and sparse with reranking

Pinecone has introduced new cascading retrieval capabilities that integrate dense and sparse retrieval methods with reranking to enhance AI search applications. These innovations aim to unify dense retrieval, which excels in semantic understanding, with sparse retrieval methods like BM25, which are effective in precise keyword matching. The new capabilities include sparse-only vector indexes and the pinecone-sparse-english-v0 embedding model, which improves precision with whole-word tokenization and increases speed by eliminating runtime inference during query encoding. Additionally, rerankers such as cohere-rerank-3.5 and pinecone-rerank-v0 further refine search results by evaluating the relevance of query-document pairs. This comprehensive approach is reported to yield significant improvements in performance, with up to 48% better results on specific benchmarks, positioning Pinecone as a leading platform for modern AI retrieval solutions.