January 2024 Summaries

The Groundedness Observability feature in deepset Cloud tracks the degree to which an answer generated by a retrieval augmented generation (RAG) system is based on the underlying documents, providing a quantifiable score for assessing the quality and error sources of LLM-powered prototypes. This feature aims to improve security and trust among users and builders by identifying hallucinations and ensuring that answers are grounded in data. The Groundedness Observability Dashboard provides insights into pipeline performance, allowing users to compare different models, optimize prompts, and refine retrieval setups. Additionally, the feature offers a Reference Predictor, which enables users to verify answers with academic-style citations, promoting confidence in LLM-powered products and paving the way for widespread adoption in production environments.

The widespread adoption of AI in 2024 will be driven by the need for scalable evaluation methods for large language models (LLMs), which was previously a major obstacle. To address this, deepset has developed an AI trust layer that provides visibility into LLMs and their output, including the Groundedness metric, which tracks the degree to which an answer is based on the underlying documents. The Groundedness metric can help improve LLM security, trust, and observability by quantifying the truthfulness of an LLM's output and identifying hallucinations. It also enables users to track the metric over time and use it to optimize their retrieval setup, reduce costs, and improve system performance.