Evaluating our AI Guard application to improve quality and control cost

Datadog's engineering teams have developed AI Guard, an application designed to monitor and secure AI agents by detecting and blocking unsafe behaviors in real time. The system uses LLM Observability to evaluate, iterate on, and monitor AI Guard, capturing every model decision and measuring the impact of changes with statistically valid datasets. AI Guard protects Datadog's Bits AI Agents by analyzing requests from user prompts to model responses, identifying potentially harmful actions such as prompt injections or unsafe code execution. The development process includes creating a comprehensive dataset that combines real and simulated agent activity, enabling accurate detection models trained on edge cases. AI Guard's performance is continuously evaluated using structured experiments, with every change to the system tracked and reviewed to ensure alignment with desired operational metrics. It currently operates on both OpenAI models and custom self-hosted models, and Datadog is exploring additional models and frameworks to expand its capabilities. In production, AI Guard integrates with Datadog's AI gateway to provide seamless evaluation and monitoring, improving the speed and efficiency of detecting and resolving issues, thus reinforcing AI system reliability.