WideSearch explained: benchmarking broad information collection for agents

WideSearch is a ByteDance Seed benchmark, released in August 2025, that evaluates AI agents’ ability to compile large, structured tables of publicly available web facts across 200 English and Chinese tasks spanning more than 15 domains. Unlike deep-search benchmarks that require finding a single obscure answer, it tests wide-search capabilities such as decomposing broad requests, tracking missing cells, validating sources, and avoiding hallucinations while collecting dozens or thousands of facts. Its strict cell-by-cell evaluation uses exact matching, normalization rules, numeric tolerances, and LLM judgment, with success requiring an entirely correct table; consequently, leading systems in the original study achieved low exact-match success rates despite substantially higher item-level F1 scores. The analysis attributes failures to incomplete planning, weak query reformulation, improper evidence use, hallucination, and formatting errors, while noting limitations including its small and uneven task set, public answer tables, aging sources, language-coverage differences, and metric inconsistencies across comparisons. Parallel’s separate 2026 evaluation of 100 tasks uses partial-credit item-level scoring rather than exact success, finding that stronger agents generally improve results more than search-provider choice, although its Advanced service led the frontier tier and Exa Auto led the lower-cost tier. The discussion also argues that production workflows may benefit from separating entity enumeration and enrichment, and provides guidance and software resources for running benchmark evaluations while clearly reporting metrics, costs, repeated trials, and language coverage.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.