Retool, an enterprise AppGen platform, has leveraged a data-driven approach to enhance its AI-powered development assistant, Assist, by using production logs as a primary input for roadmap decisions. By employing Braintrust's AI assistant, Loop, Retool queries production data semantically, enabling quick insights and facilitating the prioritization of improvements based on user needs and impact. This shift from an intuitive to a data-informed strategy allows Retool to maintain quality while scaling its user base, as evidenced by the successful prioritization and implementation of features like multi-page support. The team categorizes user intents through a classifier agent, analyzes production data using BTQL, and monitors critical issues with internal dashboards to ensure that engineering efforts align with user demands. Key insights, such as the high demand for multi-page app support and AI-powered README functionalities, were identified through this approach, allowing Retool to address significant user needs efficiently. This transformation underscores the importance of systematic observability in AI development, where production data not only informs debugging but also guides product development, ensuring that resources are directed toward initiatives with the most substantial user impact.