Improving Zero-Shot Ranking with Vespa Hybrid Search - part two

The blog post explores the enhancement of zero-shot ranking using Vespa Hybrid Search by evaluating various ranking models on the BEIR benchmark in a zero-shot setting. It introduces a new BM25 baseline for the BEIR dataset, which surpasses previously reported results, and demonstrates that a hybrid approach combining BM25 with a neural ranking method outperforms other evaluated methods on 12 of 13 datasets. The hybrid model's effectiveness is further compared to emerging few-shot methods that use large language models to generate synthetic training data, highlighting that while few-shot models perform better, the Vespa hybrid model offers a cost-effective alternative without needing in-domain adaptations or extensive computational resources. The post also emphasizes the importance of establishing a strong BM25 baseline to avoid overestimating neural ranking progress, particularly in zero-shot settings where single vector representations face generalization challenges. The Vespa app supporting this research is open-source, accessible for reproduction, and can be deployed both on Vespa Cloud and locally.