HHEM | Flash Update: Fast. But Are They Furious?

This week witnessed significant developments in the AI field, with major announcements from OpenAI and Google regarding their latest models. OpenAI introduced GPT4o, a faster and cheaper model than GPT4-Turbo, noted for its omni-modal capabilities, while Google announced the general availability of Gemini 1.5 Flash, which retains the long context length of previous models. Both models were evaluated using the Hughes Hallucination Evaluation Model, revealing that although they are faster, they exhibit higher hallucination rates compared to their predecessors. GPT4o's hallucination rate increased to 3.7%, and Gemini 1.5 Flash worsened to 5.3%, illustrating a trade-off between speed, cost, and model performance. Additionally, the AI community received news of Ilya's departure from OpenAI, acknowledging his pivotal contributions to the field.