llms.txt

llms.txt is a lightweight convention borrowed from the robots.txt pattern. You create a plain text file at yourdomain.com/llms.txt and use it to tell AI systems, crawlers, and large language model (LLM) assistants exactly which pages matter most on your site, what your product does, and how your content is organized. Unlike robots.txt, which restricts access, llms.txt is designed to actively invite and guide AI systems toward your best content.

The file typically opens with a short description of the site or product, followed by a structured list of URLs grouped by topic, along with brief annotations explaining what each section contains. Some implementations include a prose summary at the top so that AI assistants ingesting the file can immediately understand who you are and what you offer, even before following any links.

For AI visibility and generative engine optimization (GEO), llms.txt matters because AI systems often need to quickly decide which parts of a site are authoritative and worth citing. A well-structured llms.txt removes ambiguity. It signals that your most important pages are the product overview, documentation, pricing, and key blog posts rather than legal pages or internal search results. Whether it actually moves your citation rate is a separate, unresolved question: checking the 50 most-cited domains in one category found the 29 that publish an llms.txt file rank no better than the 21 that don't, and the two most-cited domains of all have no file at all.

To create one, start with a one-paragraph plain-English description of your site, then list your top pages grouped by category (product, docs, blog, use cases) with a short annotation on each line. Keep it under a few hundred lines, update it when you publish major new content, and verify that AI crawlers are not blocked from fetching it in your robots.txt. See the full llms.txt guide for exactly what belongs in each section.