llms.txt and llms-full.txt: useful context without false promises

What the format is

llms.txt is a public format proposal for giving language models a concise Markdown description of a site and links to useful material. It is not a robots directive and does not replace HTML, sitemap, canonical URLs, or structured data.

Publishing the file does not guarantee that a model will fetch it, index the site, cite the site, or prefer it in an answer. AI Web Check therefore treats it as experimental coverage rather than a scored AI Readiness requirement.

What AI Web Check evaluates

The checker looks for public /llms.txt and /llms-full.txt responses and reports their availability separately from the main score. It does not recursively crawl every link listed in those documents.

This keeps the distinction between discovery context and permission to perform additional network or agent actions.

A practical minimal shape

Keep llms.txt short: identify the service, describe its purpose, and link to the strongest public pages. Use llms-full.txt only when additional context about capabilities, limitations, or machine-readable interfaces is genuinely useful.

# Example Service

> Public technical documentation and machine-readable interfaces.

## Key pages
- [Documentation](https://example.com/docs)
- [API contract](https://example.com/openapi.json)

## Limits
- This file does not promise indexing, ranking, or citation.

Common mistakes

• Turning the file into a long SEO text instead of concise machine navigation.
• Publishing private endpoints, tokens, internal repository URLs, or operational secrets.
• Allowing llms.txt to contradict canonical public pages or stale documentation.
• Claiming that the presence of the file alone makes a site “AI optimized”.