Home / Blog / Improving Agent Readiness with Website …

Improving Agent Readiness with Website Spec

Human-written post Startup GTM Guidance

July 6, 2026 by Matt Makai

If you want to improve the performance, SEO/GEO, security, accessibility, and more (who wouldn't want to do that?) of your website, you should go to one of the pages on Website Spec then use the following prompt in Codex, Claude Code, or Cursor while in your codebase. This example is for agent readiness:

Check the current Plushcap implementation against this set of agent readiness specifications and report on what the application does well, where it needs improvement, and what you recommend we fix immediately? https://specification.website/spec/agent-readiness/

Your AI coding agent of choice will churn through the specifications one by one, go through your implementation, then come up with an action plan. Here's the one that mine put out over the weekend if you want to see what type of output you'd get with Codex + GPT-5.5 before trying it yourself.

Note that the rest of this blog post is output from GPT-5.5, while this line and everything before it are human-written like the rest of this blog. Agent readiness output example for Plushcap:

Plushcap Agent-Readiness Assessment And Improvement Plan

Assessment date: July 6, 2026

Spec reviewed: https://specification.website/spec/agent-readiness/

Summary

Plushcap is already ahead of a typical site on agent readiness. It has server-rendered HTML, canonical URLs, sitemap coverage, /llms.txt, Markdown docs endpoints, AI crawler rules, an Agent Skills index, an AI catalog, a public REST API, and a real MCP server with OAuth, server card, tool schemas, and read-only annotations.

The biggest gaps are discovery polish and consistency. Production static pages bypass Django middleware, so important pages like /, /docs, and /companies currently do not send the Link discovery headers that the spec expects on the homepage. This was verified with curl -I against production on July 6, 2026.

What Plushcap Does Well

  • /llms.txt is live and useful. The endpoint is concise, points agents to docs, API resources, MCP, Agent Skills, and important public JSON API endpoints, and is served as text/markdown.
  • Agent Skills discovery is implemented. /.well-known/agent-skills/index.json includes the v0.2.0 schema, a plushcap-consumer skill, a public Markdown URL, CORS, and a matching SHA-256 digest.
  • Docs have Markdown source endpoints. /docs.md and /docs/<path>.md return Markdown with text/markdown; charset=utf-8 and CORS.
  • robots.txt explicitly names major AI crawlers. It includes GPTBot, Google-Extended, Applebot-Extended, ClaudeBot, anthropic-ai, Bytespider, CCBot, OAI-SearchBot, ChatGPT-User, and PerplexityBot, plus Content-Signal directives.
  • Sitemap coverage is broad. The sitemap system covers companies, static pages, docs, learn pages, trends, insights, public trend reports, blog posts, and optional large content-post sitemaps.
  • MCP is a strong implementation. Plushcap has an MCP server on mcp.plushcap.com with OAuth protected-resource metadata, a server card, streamable HTTP transport, tool listing, read-only annotations, and output schemas.
  • The public site is mostly server-rendered. Key page content is present in initial HTML without requiring JavaScript. Some charts are client-rendered, but pages generally include headings, summaries, tables, and links in the HTML response.
  • JSON-LD exists globally. Pages can emit Organization, WebSite, and breadcrumb structured data through the shared SEO helper.

Where Plushcap Needs Improvement

  • Production static pages miss discovery Link headers. Django middleware adds discovery headers, but nginx serves distilled static HTML before requests reach Django. As a result, production pages such as /, /docs, and /companies do not include the discovery headers.
  • Current Link relation names need cleanup. The middleware uses relation names such as rel="llms" and rel="ai-catalog". The spec prefers registered relation types where possible, especially describedby, alternate, sitemap, api-catalog, and service descriptors.
  • Markdown content negotiation is incomplete. Docs support .md URLs, but canonical docs URLs such as /docs and /docs/mcp still return HTML when requested with Accept: text/markdown.
  • HTML does not advertise Markdown alternates. Docs pages should include <link rel="alternate" type="text/markdown" href="...md"> in <head>.
  • There is no /llms-full.txt. This is optional, but Plushcap's public docs set is small enough that a generated concatenated Markdown file would be practical.
  • Structured data is too generic. Most pages only get site-level Organization, WebSite, and breadcrumb schema. Blog posts, trend reports, company pages, docs, and trend pages should use more specific schema.org types.
  • robots.txt sends mixed intent for training crawlers. Several crawler groups combine Disallow: / with Content-Signal: search=yes, ai-input=yes, ai-train=yes. If the intent is to block training crawlers, use ai-train=no; if the intent is to allow /content/ only, make that explicit and test validator behavior.
  • Feeds are missing or not discoverable. Plushcap has JSON APIs, but no obvious RSS, Atom, or JSON Feed endpoint for recent blog posts, insights, reports, or content updates.
  • Static deployment risks stale discovery behavior. Static HTML, static XML, and dynamic Django responses do not currently have identical discovery headers or content negotiation behavior.

  1. Add discovery Link headers at nginx or Cloudflare for static pages. This is the highest-priority spec miss because production homepage discovery currently fails even though Django middleware has the right idea.

  2. Normalize discovery Link relations. Use shapes like:

http Link: </llms.txt>; rel="describedby"; type="text/markdown"; title="Site index for LLMs" Link: </sitemap.xml>; rel="sitemap"; type="application/xml" Link: </.well-known/agent-skills/index.json>; rel="agent-skills"; type="application/json"

Keep MCP/API catalog relations aligned with the closest registered relation names available.

  1. Add Markdown discovery on docs pages. Add <link rel="alternate" type="text/markdown" href="...md"> to docs HTML and include the Markdown alternate in page-level Link headers.

  2. Support Accept: text/markdown for docs. Canonical docs URLs should return Markdown when requested with Accept: text/markdown, and include Content-Location plus Vary: Accept.

  3. Clarify the AI crawler policy in robots.txt. Make Disallow and Content-Signal agree. For example, if training is not allowed, use Content-Signal: search=yes, ai-input=yes, ai-train=no for training crawler groups.

  4. Add page-specific JSON-LD for high-value pages. Start with blog posts, trend reports, company profile pages, docs, and trend pages.

  5. Add a simple feed. Start with RSS or JSON Feed for Plushcap blog posts or public trend reports, then advertise it through HTML <link rel="alternate">, HTTP Link, and /llms.txt.

Lower-Priority Improvements

  • Generate /llms-full.txt from the public docs Markdown files and link it from /llms.txt.
  • Add an OpenAPI schema endpoint for /api/v1/ and advertise it from docs, Link headers, /llms.txt, and the AI catalog.
  • Add JSON alternatives for important public listing pages where the REST API does not already cover the same job.
  • Add schemamap-style JSON-LD endpoints only after page-specific JSON-LD coverage is broad enough to justify indexing.
  • Consider A2A, NLWeb, WebMCP, or DNS-AID only if Plushcap wants to expose an autonomous assistant or browser-native tool surface beyond MCP.

Verification Notes

Production checks run on July 6, 2026:

  • https://www.plushcap.com/llms.txt returned 200 with content-type: text/markdown; charset=utf-8.
  • https://www.plushcap.com/.well-known/agent-skills/index.json returned 200 with content-type: application/json.
  • https://www.plushcap.com/robots.txt returned 200 with content-type: text/plain.
  • https://www.plushcap.com/docs.md returned 200 with content-type: text/markdown; charset=utf-8.
  • https://www.plushcap.com/ returned 200 HTML, but no discovery Link header.
  • https://www.plushcap.com/docs returned 200 HTML, but no discovery Link header.
  • https://www.plushcap.com/companies returned 200 HTML, but no discovery Link header.
  • https://www.plushcap.com/docs with Accept: text/markdown still returned HTML.

Local Django test-client checks showed the dynamic discovery endpoints and Django middleware are functioning, but production static delivery bypasses the middleware for distilled pages.

Test Plan

  • Add tests for /llms.txt, /.well-known/ai-catalog.json, /.well-known/agent-skills/index.json, the public SKILL.md, /docs.md, and /docs/mcp.md.
  • Add a digest verification test for every Agent Skills index entry.
  • Add integration or deployment smoke checks for production headers on /, /docs, /companies, /llms.txt, and /.well-known/agent-skills/index.json.
  • Add tests for Accept: text/markdown, Content-Location, and Vary: Accept once docs content negotiation is implemented.
  • Add tests for page-specific JSON-LD by page type before broad rollout.
  • Add feed validation once RSS, Atom, or JSON Feed is introduced.

Source References

  • Agent readiness: https://specification.website/spec/agent-readiness/agent-readiness-overview/
  • /llms.txt: https://specification.website/spec/agent-readiness/llms-txt/
  • Markdown source endpoints: https://specification.website/spec/agent-readiness/markdown-source-endpoints/
  • Robots for AI crawlers: https://specification.website/spec/agent-readiness/robots-for-ai-crawlers/
  • HTTP Link headers: https://specification.website/spec/agent-readiness/link-headers/
  • Structured data for agents: https://specification.website/spec/agent-readiness/structured-data-for-agents/
  • Machine-readable formats: https://specification.website/spec/agent-readiness/machine-readable-formats/
  • MCP and tool discovery: https://specification.website/spec/agent-readiness/mcp-and-tool-discovery/
← Back to Blog
Plushcap, by Matt Makai. 2021-2026.