Glossary
What is llms.txt? The AI Robots.txt Explained
llms.txt is a plain text file placed at the root of a website (example.com/llms.txt) that provides AI language models with a structured summary of the site, including its purpose, key pages, content categories, and machine-readable metadata.
While robots.txt tells search engine crawlers what they can and cannot access, llms.txt tells AI language models what your site is about and which pages are most important. It is the AI equivalent of a site introduction: a concise, structured document that helps ChatGPT, Claude, Perplexity, and other AI systems understand your content before they process it.
llms.txt vs robots.txt
robots.txt controls crawl access: which bots can crawl which URLs. llms.txt is informational: it describes your site to AI models so they can better understand and cite your content. They serve complementary purposes. A site should have both.
What to Include in llms.txt
A strong llms.txt file includes: a one-paragraph site description, your 10-20 most important pages with one-line summaries, content categories (blog, docs, product pages), key facts about your business (pricing, features, unique capabilities), and any specific instructions for how AI should represent your brand. Keep it under 100 lines.
How llms.txt Improves AI Visibility
AI crawlers that find an llms.txt can more quickly understand your site structure and prioritize your best content. This is especially useful for sites with thousands of pages where AI crawlers might otherwise spend time on less important pages. It also reduces the chance of AI hallucinating incorrect facts about your brand by providing ground truth.
llms.txt Validation
Pinealizer's AIO audit includes an llms.txt collector that checks whether your file exists, validates its format, and scores its completeness. The audit flags missing sections, overly long files, and pages listed in llms.txt that return errors.
FAQ
llms.txt FAQ
llms.txt is a plain text file at your website root that helps AI language models understand your site. It includes a site description, key pages, and important facts. Think of it as robots.txt for AI comprehension.
If you want AI systems like ChatGPT and Perplexity to accurately represent your brand, yes. It is not required but strongly recommended. Sites with llms.txt give AI a curated starting point instead of letting it guess.
Create a plain text file at your-domain.com/llms.txt. Start with a one-paragraph site description, list your most important pages with summaries, and include key facts. Pinealizer validates your llms.txt automatically during audits.
Google AI Overviews do not officially use llms.txt, but GPTBot (ChatGPT), ClaudeBot, PerplexityBot, and other AI crawlers may use it. Having one ensures your site is prepared for current and future AI systems.
Related
Related terms
Ready?
Check your AI visibility
Run a free scan. See how your site performs across all 26 disciplines.