New Relic AI aims to unlock observability for all by enabling users to interact with their telemetry data using natural language, reducing reliance on New Relic Query Language (NRQL) proficiency and streamlining analysis across tech stacks. To evaluate the performance of this system, the development team identified areas where clear-cut measures of success can be applied, such as classifying user requests into different flows and performing syntax validation in NL2NRQL. They also tackled hallucinations by using retrieval-augmented generation (RAG) and reframing the task to ensure the model relies on external context rather than built-in knowledge. Additionally, they promote trustworthiness via transparency and clear communication throughout the Q&A flow, providing users with intermediate steps and explanations to assess the validity of answers. By implementing these strategies, New Relic AI can deliver real value to users while minimizing potential pitfalls.