Do Smaller Models Hallucinate More?

In the realm of retrieval augmented generation (RAG) applications, reducing hallucination rates remains crucial, as demonstrated by the widespread adoption of the Hughes Hallucination Evaluation Model (HHEM) since its launch in November 2023. Vectara's integration of the Factual Consistency Score into their RAG pipeline and the collaboration with Intel highlight the industry's advancements in ensuring accuracy and reliability. Intel's Neural Chat 7B model, optimized with their Gaudi©2 processor and Direct Preference Optimization, has achieved a notable 2.8% hallucination rate, outperforming larger models like GPT-4. This progress underscores the potential of smaller models to excel in specific tasks such as reducing hallucinations, challenging the traditional preference for larger models due to their costly and slow inference processes. Vectara's HHEM has become a standard for hallucination detection, and the improvements in LLMs' ability to consistently summarize facts enhance the overall performance and trustworthiness of RAG applications, marking a significant contribution to the AI community.