July 2024 Summaries

The Galileo's Hallucination Index, RAG Special, is a new benchmarking study that evaluates the performance of leading foundation models in real-world Retrieval-Augmented-Generation (RAG) use cases. The study tested 22 models from top brands like OpenAI, Anthropic, and Meta, measuring their context length impact on model performance. It also explored the open-source vs. closed-source software debate, with results suggesting that closed-source models may not necessarily perform better than open-source ones. The study used Galileo's Context Adherence Evaluation Model to measure LLM adherence to provided information, helping identify instances of hallucinations.

Today, Galileo is excited to announce its partnership with HP, a collaboration that brings together two industry leaders committed to accelerating the development of accurate and trustworthy AI. As part of this alliance, Galileo will integrate into the Z by HP AI Studio, making it easier for users to build, evaluate, monitor, and protect AI applications. Enterprise teams globally have adopted Generative AI at an incredible pace, but many are grappling with hallucinations, unpredictable behavior, and security concerns that prevent their LLM powered applications from being reliable. The integration of Galileo's comprehensive end-to-end evaluation and observability platform will offer tools for rapid experimentation, real-time monitoring, and proactive protection against harmful outputs, enabling faster and safer deployment of AI-powered applications. With this partnership, HP becomes the only manufacturer to develop an AI platform for workstations with built-in generative AI trust for LLM development. The Z by HP AI Studio with Galileo integration will be available in the fall of 2024.