We measured time-to-first-token on 275,000 real agent requests. The benchmarks lied.

Analysis of roughly 275,000 live coding-agent requests through the Weave router finds that vendor latency benchmarks often differ from production performance because real workloads involve large contexts, varied tasks, and routing patterns. The data shows that latency has distinct components—time to first token, sustained output throughput, and total completion time—that favor different models and workloads: DeepSeek v4-flash streams rapidly, GPT-5.5 has similarly quick initial responses but lower throughput, Claude models incur slower startup times while maintaining competitive generation speeds, and DeepSeek v4-pro achieves low total latency through concise outputs. The figures are not controlled measures of intrinsic model speed because models receive different request types, illustrated by Gemini Flash Lite’s low apparent throughput on short label-generation tasks. The central conclusion is that no model is universally fastest, and effective routing should match models to task characteristics, distinguishing interactive requests that prioritize first-token speed from agent workflows that depend on full-response throughput and completion time.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.