HHEM | Flash Update: Phi 2

Microsoft Phi 2 has been added to a hallucination leaderboard, which measures the factual consistency of large language model (LLM) generated summaries. This leaderboard is relevant for models used in retrieval-augmented generation (RAG) architectures, where the role of the LLM is to interpret retrieved information, minimizing hallucinations. As of January 16, 2024, Phi 2 shows an 8.5% hallucination rate, comparable to other models like Cohere and Claude 2, and better than Mixtral 8x7B and Titan Express. Despite having only 2.7 billion parameters, Phi 2's efficient training allows it to outperform larger models, facilitating broader adoption due to lower hardware requirements and cost. Additionally, Microsoft's decision to move Phi 2 to the MIT license on January 5th simplifies its integration into commercial and enterprise systems, increasing competitive pressure on proprietary LLM vendors and ultimately benefiting consumers through faster innovation and reduced prices.