How we cut GitLab Duo Code Review’s cost by 98%

A study of GitLab Duo Code Review on Martian’s public 50-pull-request benchmark found that combining model selection with context compaction reduced estimated per-review token costs from $0.589 to $0.0135, a 97.7% decrease, while F1 remained near the baseline at 47.2% versus 46.4%. Duo’s four-stage workflow separates repository exploration, file fetching, analysis, and final review, so optimization required examining each stage rather than applying generic agent techniques designed for a single growing conversation. Compacting changed-file windows and converting verbose diffs to unified format reduced median review prompts from 59,000 to 32,000 tokens, but by itself saved only 19.4% because the baseline model generated more reasoning output. Replacing all stages with the self-hosted open-weight DeepSeek V4 Flash cut costs by 85.3%, but some oversized prompts exceeded its 256,000-token deployment limit and fell back to the more expensive Claude Sonnet 5; compaction eliminated those fallbacks by reducing the largest prompt from 480,000 to 109,000 tokens. A lower-cost Claude model worked for mechanical exploration and fetching stages but reduced quality in analysis and review, while higher reasoning effort and prompt caching offered limited value. The authors emphasize that results were tuned and measured on the same public benchmark, use published token prices rather than infrastructure costs, may not generalize to other codebases, and illustrate the broader value of measuring costs by stage and retesting interacting harness changes.

No tracked trend matches for this post yet.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.