Showing our work: a controlled experiment in context engineering

dbt Labs evaluated whether AI agents could retrieve customer-call context more efficiently from warehouse-modeled data through dbt MCP rather than directly from Gong’s MCP connector. In an 80-run controlled experiment using a sales recap PDF skill, the team compared full Gong transcripts, full transcripts queried from Snowflake through dbt MCP, short Gong-generated briefs, and internally engineered AI summaries. Querying full transcripts from the warehouse was about 14% cheaper and faster than Gong because structured warehouse fields enabled more precise filtering, while Gong briefs were the cheapest and fastest option but omitted nearly all key details such as dates, numbers, roles, decisions, and next steps. Internally generated summaries, created with Snowflake AI functions and tailored extraction rules, reduced cost by 48.5% and runtime by 43.2% while retaining most dates, numbers, tools, and next steps, though they still missed some titles and decisions. The findings support treating AI context as modeled data: centralizing, versioning, and improving curated context in the warehouse can reduce vendor API pressure, eliminate duplicated retrieval logic, and improve multiple AI skills simultaneously.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.