Company
Date Published
Author
Charlie Hull & Matthias Krueger
Word count
1441
Language
English
Hacker News points
None

Summary

We evaluated the performance of Vectara's new AI-powered search engine, Boomerang, against a popular question/answering model, USE-QA. We tested Boomerang on five datasets for different applications, including e-commerce search and scientific search, and found that it provided superior performance in four of the five datasets. The evaluation also highlighted areas where cross-language information retrieval could be improved. We suggested integrating Quepid with Vectara to test and tune its engine further, which is now live on www.quepid.com. Additionally, we provided feedback on available features and suggested upgrades, such as an "upsert" capability and more useful metadata filters. The Vectara team is using our results to improve Boomerang, demonstrating their commitment to testing and evaluation.