HHEM | Flash Update: Google Gemma

Google has introduced Gemma, a new open-source large language model (LLM) available in 2B, 7B, and instruction fine-tuned versions, following the release of Gemini 1.5. Gemma's hallucination rate, measured at 7.5% using the Hughes Hallucination Evaluation Model (HHEM), places it alongside Cohere’s Chat model, slightly below Llama2 13B, but better than Mistral 7B at 9.4%, although not as low as Llama2 7B's 5.6%. With a perfect answer rate of 100.0%, Gemma is particularly effective in summarization tasks, making it a strong candidate for integration into commercial and enterprise systems, facilitated by its liberal "Gemma Terms of Use." This release signifies a broader industry trend towards open licensing, reminiscent of Microsoft's move with Phi 2, as proprietary LLM vendors face increasing competition to innovate and offer competitive pricing. The release of Gemma underscores the intensifying competition for efficiency, performance, and accessibility in the LLM sector, ultimately benefiting end users.