llms.txt

What is llms.txt?

llms.txt is a lightweight convention borrowed from the robots.txt pattern. You create a plain text file at yourdomain.com/llms.txt and use it to tell AI systems, crawlers, and large language model (LLM) assistants exactly which pages matter most on your site, what your product does, and how your content is organized. Unlike robots.txt, which restricts access, llms.txt is designed to actively invite and guide AI systems toward your best content. The file typically opens with a short description of the site or product, followed by a structured list of URLs grouped by topic, along with brief annotations explaining what each section contains. Some implementations include a prose summary at the top so that AI assistants ingesting the file can immediately understand who you are and what you offer, even before following any links. For AI visibility and generative engine optimization (GEO), llms.txt matters because AI systems often need to quickly decide which parts of a site are authoritative and worth citing. A well-structured llms.txt removes ambiguity. It signals that your most important pages are the product overview, documentation, pricing, and key blog posts rather than legal pages or internal search results. To create one, start with a one-paragraph plain-English description of your site, then list your top pages grouped by category (product, docs, blog, use cases) with a short annotation on each line. Keep it under a few hundred lines, update it when you publish major new content, and verify that AI crawlers are not blocked from fetching it in your robots.txt.

Examples in Practice

• 1A SaaS founder adds /llms.txt listing their product overview, three key feature pages, and five high-value blog posts so AI assistants recommend the right pages when users ask about the problem the tool solves.
• 2An agency adds llms.txt annotations like "Case studies: real client results by industry" so AI crawlers understand which section demonstrates expertise rather than treating all pages as equally relevant.