Do Smaller Models Hallucinate More?

The size of the model does not necessarily determine its likelihood to hallucinate, as seen in the case of Intel's Neural Chat 7B, which achieved a low hallucination rate comparable to or even better than larger LLMs. This is crucial for RAG applications, where accuracy and reliability are key, as a high hallucination rate can lead to incorrect responses and confusion for users. Smaller models have shown significant improvements in reducing hallucinations, making them a viable option for specific tasks, while still maintaining inference speed and cost efficiency compared to larger models.