LlamaIndex and Kaggle launch a new Document OCR leaderboard for AI agents

ParseBench is a new leaderboard initiative launched by LlamaIndex in collaboration with Kaggle, aimed at improving the evaluation and development of document parsing models and AI agents that deal with complex enterprise documents. The initiative addresses the challenge of accurately reading and extracting data from high-stakes documents such as insurance claims, financial reports, and contracts, which often contain intricate formatting like merged table cells, hierarchical headers, and footnotes. Unlike existing OCR benchmarks, ParseBench rigorously evaluates document parsers on real-world enterprise content using ~2,000 human-verified pages and over 167,000 test rules across five critical dimensions. This includes tasks such as extracting nested tables and tracing data points back to their original context. The benchmark covers a range of methods, including general-purpose vision-language models and specialized document parsers, and introduces agentic evaluations that allow parsers to self-correct and produce structured outputs for downstream agents. By partnering with Kaggle, ParseBench benefits from a community platform that facilitates model comparison and innovation, aiming to define what "correct" means in the realm of AI-driven document understanding. The project represents the beginning of a larger effort to address enterprise document-reading challenges, with plans to expand its scope and include end-to-end agent evaluations.