HANDBOOK.md: Can Agents Follow 100-Page Company Policies?

HANDBOOK.md is a benchmark designed to evaluate AI agents' ability to follow complex, realistic company handbooks in live enterprise environments, spanning domains such as Finance, Medical Billing, Insurance, Logistics, and HR. The benchmark challenges AI by requiring them to navigate tasks with extensive policy documents, which can range from 20 to 124 pages and come in various formats like PDF and Word. Despite these detailed instructions, current frontier AI models, including GPT-5.5 and Opus 4.8, struggle with these tasks, often failing to comply with crucial rules and achieving success rates below 25%. The benchmark highlights critical failure patterns, such as prioritizing immediate requests over standing policies and losing track of information over extended tasks. This reflects the ongoing challenge for AI models to adhere to complex, long-standing instructions in dynamic, real-world settings.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.