DeepSearchQA explained: how deep research agents are scored

DeepSearchQA is a Google DeepMind benchmark released in December 2025 that evaluates deep-research agents on 900 time-anchored prompts across 17 domains, with most questions requiring complete sets of answers rather than a single fact. It is designed to measure agents’ ability to gather fragmented information, resolve duplicate entities, apply multi-step constraints, and determine when searching is complete, addressing what its authors call the “Comprehensiveness Gap.” Performance is primarily assessed through per-prompt precision, recall, and F1 scores, alongside the stricter share of fully correct answers; results show that leading agents often achieve high F1 while still missing or adding individual list items, described as the “Last Mile Problem.” The public dataset contains 584 set-answer and 316 single-answer prompts and uses an LLM-based equivalence judge, making scores sensitive to the selected grading model and prompt. The text distinguishes between end-to-end deep-research-agent benchmarks, where Parallel reports high accuracy on a 100-question subset, and Artificial Analysis search-tool evaluations on all 900 prompts, where Exa leads DeepSearchQA F1 and Parallel’s advanced search ranks alongside Perplexity. It also notes limitations including public-answer contamination risk, no held-out split, potential source changes, and the fact that outcome-only grading cannot distinguish careful research from lucky guesses.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.