The llms.txt format is a small, voluntary Markdown file that lists a site's most useful pages in a fixed order, and the llms.txt Generator builds that file locally from your own inputs without crawling your domain. The format proposed at llmstxt.org asks for one required H1 holding the project or site name, an optional blockquote summary, optional non-heading details, and zero or more H2 sections whose Markdown list items each link to a canonical resource. The order is strict: the H1 comes first, the summary follows when present, free-form details appear before any H2 link sections, and a dedicated H2 named Optional marks secondary material that can be skipped when a shorter context is needed. Because the file is meant to be predictable for both language models and ordinary parsers, that order matters, and an llms.txt Generator owns it instead of accepting arbitrary Markdown fragments. The tool also rejects unsafe URL schemes, normalizes Markdown-sensitive text, and detects duplicate normalized targets before serialization, so the draft you copy or download is ready to review as content rather than as a mechanical SEO artifact.
What llms.txt Actually Proposes and Where It Stops
An llms.txt file is an emerging proposal, not a robots directive, a security control, a sitemap replacement, or a guaranteed discovery protocol. Publishing one does not force a crawler or assistant to request it, cannot grant access to blocked pages, cannot override authentication, and cannot prove that an AI answer will cite the site. Treating it as one more curated mechanism that complements existing web controls keeps expectations honest. Crawl preferences for participating crawlers live in robots.txt. Indexable URL inventories for search discovery live in sitemap.xml. Entity and page descriptions live in structured data. llms.txt offers a small Markdown overview and a short list of high-value references for inference-time use, and each mechanism should keep its own purpose rather than be folded into another. Editors who want to read the syntax directly can start with the proposal's implementation documentation at DeveloperHub and the AnswerDotAI/llms-txt reference repository, which together describe the proposed structure and its conventions.
Inside the llms.txt Generator (Inputs, Output, Local Processing)
The llms.txt Generator is a purely client-side form that serializes the format above from inputs you supply. It never scans a domain or sitemap because a form cannot prove which dynamic pages are canonical, current, accessible, or safe to recommend. You enter a site or project name, an optional summary, optional explanatory details, and grouped links to the most useful resources, then receive deterministic plain text that you can copy or download as llms.txt. The generator owns the Markdown order rather than accepting arbitrary fragments that could produce duplicate top headings or misplaced sections. URLs are reviewed before serialization: HTTP and HTTPS links are accepted, while credentials, executable schemes, and embedded-data schemes are rejected. Labels, notes, headings, and summaries are normalized so control characters or accidental Markdown delimiters cannot break the generated structure, and duplicate normalized URLs are reported instead of silently multiplying entries.
| Element | Required? | Position in file | Purpose |
|---|---|---|---|
| H1 with site or project name | Required | First line | Identifies the project |
| Blockquote summary | Optional | Directly under the H1 | One-sentence description of the site |
| Free-form details | Optional | After the summary, before any H2 | Explanatory paragraphs about the project |
| H2 sections with link lists | Optional, zero or more | After details | Grouped canonical resources |
| H2 named Optional | Optional | Among H2 sections | Secondary material skippable in shorter context |
Build the llms.txt File With the Generator
The generator follows a small, repeatable path from form input to downloaded file. Each step is an editorial action rather than an automatic crawl, so the resulting file reflects the resources you actually want surfaced.
- Open the llms.txt Generator and enter the site or project name exactly as it should appear in the H1, since that line is the only required element of the file.
- Add an optional blockquote summary that describes the site in one sentence, and add free-form details only if a short paragraph of context genuinely helps a model understand the project.
- Add only canonical, high-value links under clear H2 section headings. For each link, use a stable URL, a short label, and an optional colon-separated note explaining what the page contains; the generator will not invent .md URLs or scan your sitemap.
- Click Generate to produce deterministic Markdown. Read the Optional section carefully, because the tool does not automatically label a resource optional, so you decide which sources are essential for understanding the site and which are genuinely secondary.
- Copy the result or download it as llms.txt, then review every line as content: confirm the title and summary match the site, every link is canonical, notes are factual, optional resources are genuinely secondary, and no private or staging URL appears.
- Switch to validation mode by pasting the final edited draft back into the validator. Confirm there are no concrete errors, then publish the file at /llms.txt or the appropriate subpath with a plain-text or Markdown-compatible content type.
- Periodically verify that each linked URL still returns the intended public content, and recheck the proposal and reference repository before building any automation that depends on exact syntax.
Validate a Draft Before You Publish
Generation and validation are deliberately separate. The validator accepts a pasted llms.txt draft and reports concrete, line-oriented issues and warnings without rewriting the file behind your back, without following any listed URL, and without claiming that the linked resource is accurate. It checks for the required H1, the documented heading order, link-list syntax, duplicate targets, and bounded size. Validation success means the draft matches the tool's documented interpretation of the current proposal; it does not certify broader vendor support, because the proposal and ecosystem can still evolve. Use the validator when you have hand-edited a generated draft, when you are copying from a template elsewhere, or when a teammate has sent a draft for review. If the validator reports duplicate normalized URLs or duplicate H1s, decide which entry is the canonical one and remove the rest before publishing. If it reports control characters or unsafe URL schemes, fix the source text rather than asking the generator to silently clean it. Treating validation as a structural check keeps the human editorial review separate from the format check.
How llms.txt Fits Next to robots.txt and sitemap.xml
Because llms.txt is a new mechanism sitting alongside familiar ones, it helps to place it in a single comparison. Curation matters more than length. A file that lists every page can repeat the overload of a large sitemap and waste a consumer's context, so prefer canonical documentation, product explanations, policies, and stable reference pages. Link to Markdown versions when the site reliably serves them, but do not invent .md URLs that return errors. If your site also needs a robots.txt file, the practical pattern for that lives in a separate practical guide to creating a robots.txt file; the two files should not be merged, since they answer different questions for different audiences.
| Mechanism | Format | Primary purpose | Voluntary adoption? | Inventories every URL? |
|---|---|---|---|---|
| robots.txt | Plain text | Express crawl preferences for participating crawlers | Yes | No |
| sitemap.xml | XML | Inventory indexable URLs for search discovery | Yes | Yes |
| Structured data | JSON-LD or Microdata | Describe entities and page content for rich results | Yes | Per entity |
| llms.txt | Markdown | Curated overview and short reference list for inference-time use | Yes | No, deliberately curated |
Review, Publish, and Recheck Over Time
After download, treat the file as content rather than a mechanical SEO artifact. Read the H1 and summary, click every link, and confirm each note is factual. The author remains responsible for selecting sources and checking that each public URL returns the intended content, because the generator never scans a domain. Publish at /llms.txt or the appropriate subpath with a plain-text or Markdown-compatible content type, then schedule a periodic check that the file is still reachable, that every link still resolves, and that no optional resource has drifted into essential. Measurement should focus on whether your own tools and workflows actually use the file; publication alone is not evidence of search or citation improvement. Recheck the proposal and reference repository before building automation that depends on exact syntax, since the methodology and source links are visible and may evolve as the ecosystem matures.