Updated October 11, 2026. llms.txt is a proposed Markdown documentation format, not an access-control mechanism or a guarantee that every agent will read your site. The proposal was updated to v2 in August 2026.
What the file is for
Use it to give clients a concise overview and a curated path to useful documentation. Place it at the site root or a relevant documentation path according to the proposal. Availability and use depend on the client; publishing the file does not require a model or agent to consume it.
| Resource | Purpose |
|---|---|
| llms.txt | A documentation overview and selected reading links |
| robots.txt | Crawler directives; not authorization for private data |
| sitemap.xml | Preferred indexable page URLs for discovery |
| WebMCP tools | Actions exposed by a supporting browser page |
Start with a small public example
Replace the fictional URLs with working public pages. Do not list credentials, internal dashboards or private reports.
# Example Site
> Public documentation for our website and supported actions.
## Guides
- [Getting started](https://example.com/docs): Introduction and limitations.
- [Search help](https://example.com/help/search): How to search public content.
## Optional
- [Changelog](https://example.com/changelog): Historical updates.
Use accurate descriptions and keep the overview small. If you provide clean Markdown versions of the linked pages, test that those representations stay consistent with the public content. Additional full-text files are a documentation choice, not a universal agent requirement.
Deploy and check it
A static website can serve the file from its public directory. A framework or hosting platform can serve an equivalent text response. Verify the resulting URL, status and text rather than assuming that a file in your source tree is deployed.
curl --fail --location https://example.com/llms.txt
Then check each listed link and ask a compatible client to answer a specific documentation question. That is a more useful test than merely counting file lines. Update the file when its references or product descriptions change.
Relationship with WebMCP and search
A text overview does not register executable tools. Use the WebMCP implementation guide for that separate capability and keep the normal interface available.
Google's AI search guidance does not require new AI text files or special markup to appear in its AI features. llms.txt availability is not proof of indexing, ranking or traffic improvement.
What the scanner evaluates
Under ruleset 2026-10-05, B2 checks whether it can obtain llms.txt or llms-full.txt content with more than 50 characters after trimming whitespace. It is not a full format parser or proof that clients use the file. The score guide describes this product heuristic.
Run a free website check for this and other page signals. Follow up with real link checks and a client test; do not treat a pass as universal AI discoverability.