Home / Companies / Vercel / Blog / Post Details
Content Deep Dive

AGENTS.md outperforms skills in our agent evals

Blog post from Vercel

Post Details
Company
Date Published
Author
Jude Gao
Word Count
1,618
Language
English
Hacker News Points
-
Summary

A study on teaching AI coding agents framework-specific knowledge for Next.js 16 revealed that embedding a compressed 8KB documentation index in a project's AGENTS.md file achieved a 100% pass rate in coding tasks, outperforming skills that maxed out at 79% even with explicit instructions. The experiment initially tested two methods: skills, which bundle domain knowledge for on-demand use, and AGENTS.md, which provides persistent context. Skills often weren't triggered reliably, but explicit instructions improved their effectiveness. However, the unexpected success of the AGENTS.md approach was attributed to its passive context, consistent availability, and lack of decision points, which proved more effective than the active retrieval required by skills. Despite concerns about context bloat, compressing the documentation index to 8KB maintained high performance. The research suggests that while skills are useful for specific tasks, the AGENTS.md approach is currently more reliable for general framework knowledge, encouraging framework authors to provide users with an AGENTS.md snippet for improved agent performance.