How WMCHealth cut a 72-minute dbt pipeline down to 5 minutes, with one environment variable

WMCHealth analytics engineers developed a dbt pattern that supports both near-real-time operational reporting and complete historical analytics from a single codebase by using the ENV_NRT environment variable to change model behavior at runtime. Serving nine hospitals with fragmented legacy systems, the team needed five-minute refreshes for active records while retaining nightly full-dataset processing for reporting and trend analysis, but rejected larger warehouses, separate projects, and conventional incremental models because of cost, logic-drift, and performance constraints. In near-real-time mode, models process active records as tables and achieved a median runtime of 278 seconds, while the same models run overnight against the full dataset as incremental models with a 4,340-second median runtime. Narrowed data scope can propagate through downstream inner joins automatically, while unrelated models use an explicit environment-variable-based filter. The approach reduced runtime by 93.6%, cut data processed in near-real-time runs by more than 99%, and improved WMCHealth’s refresh cadence from 72 minutes to five minutes without requiring additional warehouse capacity.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.