Improved agent experience with llms.txt and content negotiation

Mintlify enhances the agent experience by using content negotiation to deliver clean Markdown documentation to agents, while maintaining human-facing HTML documentation unchanged. This approach reduces token usage by 30 times and improves discoverability for agents, as agents benefit from receiving only the semantic value needed without the overhead of unnecessary HTML elements like tags, attributes, and scripts. Mintlify also optimizes llms.txt discoverability by moving the instruction block to the top of Markdown pages and adding Link and X-Llms-Txt headers to HTTP responses, allowing agents to efficiently locate and process documentation indexes. By leveraging content negotiation, Mintlify enables a single URL to cater to both human and agent consumers, streamlining documentation management and ensuring that AI agents have access to the necessary context for effective reasoning.