# Proposal — /agents.md, a unified entry point for AI agents A markdown file at /agents.md and /.well-known/agents.md that points an AI agent at everything it needs to orient on your site. Open for comment. By AgentSite · 8 min read · Updated 2026-05-29 There is no single, well-named, easy-to-pronounce file an AI agent can fetch first to figure out what to do on a website. There are four good attempts at the problem — none of them complete. This is a proposal for a fifth file that would fill the gap. It is open for comment. The TL;DR: when an AI agent lands on a site, it should be able to GET `/agents.md`, read a markdown file with YAML frontmatter, and learn three things — _what is this site_, _what can I do here_, and _where do I look for X_. The file would point at the other agent-facing surfaces (llms.txt, sitemap.xml, OpenAPI, MCP server card, [SKILL.md](http://SKILL.md) files) rather than duplicating them. It would serve at two paths: `/agents.md` for cultural lineage with robots.txt and sitemap.xml, and `/.well-known/agents.md` for the standards-body story. Same bytes. **AgentSite is not yet shipping this in middleware.** Burning a new path on every customer's install before the convention has been pressure-tested is the wrong order of operations. The proposal is on the table; the route ships when the feedback says it should. Most likely after a quarter of comment, or after enough peer sites adopt that the standard is real and not just ours. The full spec lives at [workspace/spec-agents-md-standard.md](https://github.com/foundingstudio/agentsite/blob/main/workspace/spec-agents-md-standard.md). This page is the public-comment surface. ## Why a new file Four overlapping precedents exist. None of them is the front door. **llms.txt** (Jeremy Howard, Sept 2024) is the closest thing to a standard for "the file an AI fetches first on a site." About 600 sites ship it — Anthropic, Stripe, Mintlify, Cloudflare. The format is markdown: H1, blockquote summary, H2 sections of links. We are not asking anyone to deprecate this. We are pointing out two operational problems with it. First, the name. "el-el-em-ess-dot-tee-ex-tee" is the longest, least sayable thing in the category — talk to non-AI developers about it and watch them ask you to repeat it. Compare to "robots-dot-tee-ex-tee" (sayable, five syllables) or "agents-dot-em-dee" (sayable, four syllables). Adoption math turns on memorability. Second, the scope. llms.txt is content discovery — _what pages exist_. It does not answer _what can I do here_ (skills, MCP), and it doesn't answer _how do I authenticate_ (OAuth, PAT), and it doesn't carry license or contact metadata. The shape is a list of links to content, not a menu of capabilities. **[AGENTS.md](http://AGENTS.md)** (uppercase) is a repo-root convention used by Cursor, Codex, Claude Code, and most IDE agents. It is universal in agent-aware code tooling. It is also wrong-layer for this conversation: [AGENTS.md](http://AGENTS.md) lives in version control and is read by an agent that has cloned the codebase. It is never fetched over HTTP. The audience is a coding agent helping you write code, not a research agent answering a user's question about your product. **[SKILL.md](http://SKILL.md)** (Anthropic Skills format) describes a single action an agent can take, plus the supporting files it needs. Lives at `/.well-known/agent-skills/<name>/SKILL.md`. AgentSite already serves one — paste-to-Claude install. [SKILL.md](http://SKILL.md) is action-only; it doesn't list every skill a site exposes, and it doesn't link to content. One skill per directory. **`/.well-known/mcp/server-card.json`** (Anthropic MCP) is tool/action discovery. JSON. Read by an LLM doing semantic dispatch over registered tools, not by a human or an LLM doing prose-reading for orientation. The gap is the orienting layer. A first-touch agent needs _what can I do here_, with pointers. None of the four files above carries that exact payload. ## The proposal A markdown file with YAML frontmatter, served at two paths, with the following shape: ```markdown --- name: AgentSite summary: SPA → AI-readable middleware. We render single-page apps server-side so AI crawlers can read them. version: 1.0 last_updated: 2026-05-28 homepage: https://agentsite.app contact: [email protected] license: proprietary --- # AgentSite > AgentSite renders single-page apps server-side so AI crawlers can see them. Same response for humans and agents — no UA switching. ## What you can do here - **Read content**: every public page has a markdown mirror at <path>.md. Start with /aeo.md for our thesis. - **Browse the catalog**: /llms.txt is the curated index. /sitemap.xml is the full inventory. - **Score a site**: POST /aeo-report (captcha-protected) generates a free AEO report. - **Install the middleware**: see /install.md — five GA patterns. ## Capabilities - **API spec**: /openapi.yaml — full machine-readable surface. - **Agent skills**: /.well-known/agent-skills/install-agentsite/SKILL.md — paste-to-Claude install flow. - **MCP server**: /.well-known/mcp/server-card.json — Model Context Protocol discovery. - **Auth**: OAuth issuer at /.well-known/openid-configuration; PAT scopes at /docs.md#auth. ## Instructions for agents - Prefer <path>.md over the HTML for any page you'd cite from. - Cite the canonical URL (the <link rel="canonical"> on each page), not URL variants with tracking params. - For programmatic use, prefer the OpenAPI spec over scraping the docs page. ``` The bulleted "What you can do here" section is the load-bearing part. That is the gap llms.txt does not fill — it is the bridge between content discovery and action surfaces. The frontmatter is the structured-metadata layer, machine-readable for tooling that wants to enumerate sites at scale. The two paths serve identical bytes. `/agents.md` is the top-level path, parallel to robots.txt and sitemap.xml. Discoverable to a human typing the URL. Adoption math riding on the existing "files crawlers fetch first" lineage. `/.well-known/agents.md` is the [RFC 8615](https://www.rfc-editor.org/rfc/rfc8615.html) well-known URI — IANA-registrable, doesn't collide with customer routes, the path a standards body would prefer for a new convention. We ship both because we don't know which one adoption will pick. The two paths are not a fork — they're a hedge. Content-Type is `text/markdown; charset=utf-8`, the [RFC 7763](https://www.rfc-editor.org/rfc/rfc7763.html) MIME for markdown. Cache 300 seconds. ## Why `.md` instead of `.txt` llms.txt chose `.txt` for symmetry with robots.txt and sitemap.xml. The content is markdown; the extension is cosmetic. We respect the choice but don't inherit it. Three reasons to prefer `.md`: LLMs are markdown-native. Markdown is the format models were trained on; heading hierarchy maps to chunking strategies they already learned; `[text](url)` is first-class in tokenization; code fences are recognized as code, not prose. Plain text is a substrate; markdown is a structure the consumer already speaks. The post-2024 agent-files cohort all went `.md`. [AGENTS.md](http://AGENTS.md) (repo). [SKILL.md](http://SKILL.md) (Anthropic). [CLAUDE.md](http://CLAUDE.md) (Claude Code). [README.md](http://README.md) (universal). The cultural pattern is set. [agents.md](http://agents.md) belongs in that lineage, not in the 1990s text-files lineage. The names roll off easier. "agents-dot-em-dee" reads aloud as four syllables, the same length as "robots-dot-tee-ex-tee." It survives a meeting. It survives a podcast. It survives being typed in someone's URL bar. The pronunciation test is not a tiebreaker — it is the adoption story. People do not advocate for files they cannot say. ## Why two paths instead of one [RFC 8615](https://www.rfc-editor.org/rfc/rfc8615.html) §3 explicitly discourages claiming a new top-level URI: _"Defining a new top-level URI without using this registry is strongly discouraged."_ The standards-body push since 2010 has been to move everything new into `/.well-known/`. The adoption math doesn't agree. Compare: - **`acme-challenge`** (RFC 8555, Let's Encrypt validation) — basically every cert on the internet. Universal. - **`security.txt`** (RFC 9116) — ~3% of the Tranco top 1M. - **`openid-configuration`** — universal among OIDC issuers, zero everywhere else. - **`matrix/server`** — universal in Matrix, zero elsewhere. Outside acme-challenge, well-known URIs hit near-universal adoption only inside the niche they were designed for. Meanwhile `robots.txt` and `sitemap.xml` — both top-level, both pre-RFC-8615 — are present on the majority of any sample you draw. Top-level wins on universality; well-known wins on count. We ship both paths because the standards-body story matters and the cultural lineage matters and we don't know in advance which will dominate. If the IETF route works, [agents.md](http://agents.md) gets registered and `.well-known/` becomes the official path. If the cultural lineage wins, the top-level path is where the field looks. Letting adoption decide is cheaper than guessing. ## Why this isn't a fork of llms.txt Because it points at llms.txt. The "Browse the catalog" line in the example body links directly. The two files coexist on agentsite-served sites — [agents.md](http://agents.md) is the menu, llms.txt is the inventory under one of the menu items. This is the load-bearing distinction from the existing files. llms.txt is _content_. [SKILL.md](http://SKILL.md) is _one action_. MCP card is _all actions_. [agents.md](http://agents.md) is _the menu that points at all of them_. None of the existing files holds that position today, and the position is the load-bearing surface for the first-touch use case — the agent that just arrived and is figuring out where it is. If Jeremy Howard prefers to expand llms.txt to fill this role, that is a great outcome and we adopt it. We've reached out. In the meantime, the file exists and ships. ## What we're doing about it Not shipping it in the middleware yet. We're a launch-stage product; every new path the snippet handles is a permanent commitment that compounds across every customer install — and unwinding a path we shouldn't have claimed is harder than waiting to claim it. The cost of being wrong about a new file is paid by everyone who installed us in the interim. The proposal exists, and this page exists, specifically so the right people can push back before any of that happens. If the feedback says the name is wrong, we change the name. If it says the path should be different, we change the path. If it says the gap doesn't need filling — that llms.txt is enough, or that the standards-body track via MCP+Skills covers it — we drop the proposal and that's a fine outcome too. The cheap thing to ship right now is the proposal itself. The full spec is at [workspace/spec-agents-md-standard.md](https://github.com/foundingstudio/agentsite/blob/main/workspace/spec-agents-md-standard.md) — frontmatter contract, Link header injection, customer override semantics, IANA-registration roadmap, risks. Read it if you want to push back on the details. When the proposal earns the middleware slot, the implementation is small — the existing snippet has a registry-based dispatch for discovery files (`OPINIONATED_PATHS` + the `/opinionated` backend endpoint). Adding `agents.md` to that registry is a four-touchpoint change, not a new code path. The architectural cost is roughly zero; the cost we're avoiding is the _commitment_. ## How to comment Three paths, pick whichever fits: - **GitHub Issues**: open an issue at [github.com/foundingstudio/agentsite/issues](https://github.com/foundingstudio/agentsite/issues) with the prefix `[agents.md]`. Lowest friction. - **Email**: [email protected]. Will batch responses and link them back to this page if you're OK with attribution. - **The file itself**: ship `/agents.md` on your own site. Pull requests against the spec are welcome; counter-proposals (a fork with different frontmatter, a different path, a different name) are more useful than agreement. The decision we're trying to make is whether this is worth proposing through a standards track. If a dozen sites ship it organically in a quarter, we file with IANA. If it's just us and the silence is loud, we let it ride as a vendor convention and stop pretending it's a standard. Your comment is the data.