How We Made Our Docs Test Themselves

Stale code samples in documentation are a universal issue, especially as APIs and dependencies evolve rapidly, but automating the testing of these samples can mitigate the problem. By using the Deep Agents CLI, companies like LangChain streamline the process of making inline code testable, significantly reducing manual work. This approach involves extracting code into standalone files, adding setup and teardown scripts, and employing a CI system to regularly run and test the code snippets. Deep Agents use "skills"—reusable instructions that automate tasks like migrating inline code to standalone, testable examples, ensuring documentation remains accurate and functional. This system not only automates the laborious process but also integrates testing seamlessly into documentation workflows, improving reliability and maintainability.