Learning to Measure AI Search with Vectara

The rise in AI technology has disrupted the search engine landscape, offering a plethora of new and traditional options for users and businesses, necessitating the development of robust metrics to gauge the quality of AI-powered search engines. OpenSource Connections (OSC), with a strong background in search measurement, partnered with Vectara to evaluate its Boomerang AI-powered search model against the popular Universal Sentence Encoder QA (USE-QA), finding Boomerang superior in most tested datasets. The evaluation highlighted areas for improvement, especially in cross-language information retrieval and vague query handling. OSC also developed a Quepid integration to further enhance Vectara's search capabilities, leading to feature upgrades and bug fixes. The collaboration with Vectara, marked by transparency in sharing test results, aims to contribute to the discourse on search quality measurement and improve search engine technologies, with practical applications demonstrated in SonoSim’s advanced search interface for medical training.