Why an AI agent's revenue query was $3.5 million wrong, and how AWS fixed it with dbt Semantic Layer

AWS and dbt presented a governed architecture for AI agents querying production data that addresses the risks of agent-generated SQL, including incorrect joins, invented metric definitions, and silent filtering errors. A demonstration showed that a seemingly plausible revenue query overstated results by $3.5 million because it used pre-discount, pre-refund amounts and failed to exclude cancelled orders, pending orders, and internal test accounts, rules that existed only in the semantic layer. The proposed design uses the dbt MCP Server to map business questions to certified metrics and compile SQL through MetricFlow, while the Amazon Redshift MCP Server controls execution through read-only, single-statement restrictions, IAM permissions, and least-privilege database grants. By requiring agents to choose from defined metrics rather than invent SQL or metric names, the system aims to make fabricated metrics structurally impossible and supplies audit receipts containing compiled SQL, source models, test results, and query identifiers. The approach was demonstrated across native Redshift data, Redshift queries combining warehouse tables with Iceberg data in S3, and direct dbt execution on Athena and DuckDB, maintaining the same semantic definitions and governance regardless of the underlying execution engine.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.