The llms.txt complete guide

What is llms.txt?#

llms.txt is a plain text file placed at your website's root (yourdomain.com/llms.txt) that communicates directly with AI language models and their crawlers. Think of it as robots.txt for the AI era.

While robots.txt tells search engine crawlers what they can and can't access, llms.txt tells AI models what your site is about, which content to prioritize, and how to use your information in responses.

The format was proposed by Jeremy Howard (fast.ai) in late 2024 and adopted by a growing number of AI platforms. It gives website owners a direct channel to influence how AI models understand and cite their content.

Which AI platforms use llms.txt?#

Adoption is growing. As of early 2026, the following platforms have confirmed llms.txt support or compatibility:

Perplexity actively reads llms.txt files when crawling sites. It uses the information to understand site purpose and content priority.

Claude (Anthropic) has indicated support for llms.txt in its web crawling documentation. ClaudeBot considers llms.txt when building its understanding of a domain.

ChatGPT / GPTBot has not officially confirmed llms.txt support, but testing shows that sites with llms.txt get more accurate brand representations in responses. The file likely influences training data pipelines.

Gemini / Google processes llms.txt alongside other structured data signals. Google's documentation references it as a supplementary signal.

Open-source models (Llama, DeepSeek) depend on the training data pipeline. Sites with llms.txt tend to be represented more accurately in open-source model outputs, suggesting the files are captured during web crawling.

Even platforms that don't explicitly parse llms.txt benefit from it indirectly. The file's structured format makes your site easier for any automated system to understand.

The llms.txt file format#

The file follows a simple markdown-inspired format. It starts with a site title (H1), followed by a description, then sections for different content areas. Here's the structure broken down.

The title (preceded by #) identifies your site. Keep it short: your brand name and what you do.

The description is a paragraph explaining your site's purpose, audience, and primary content.

The sections (preceded by ##) organize your content by type. Common sections include "About", "Products/Services", "Blog", and "Documentation".

Each section contains links in the format: - Page title: Brief description

Keep the file under 100 entries. AI crawlers prioritize the first items in each section.

# BrandCited - AI Visibility Platform

> BrandCited helps brands get discovered and cited by AI search engines.
> We provide AI visibility tracking across 9 platforms, site auditing
> for AI-readiness, and growth actions to improve citations.

## About

- [About BrandCited](https://brandcited.ai/about): Company mission and team
- [How it works](https://brandcited.ai/how-it-works): Platform overview

## Products

- [AI Visibility Scanner](https://brandcited.ai/features): Track brand citations across ChatGPT, Claude, Gemini, Perplexity, Grok, DeepSeek, Llama
- [Site Audit](https://brandcited.ai/features#audit): 78 AI ranking factor checks
- [Growth Actions](https://brandcited.ai/features#actions): Prioritized fixes with code snippets
- [Pricing](https://brandcited.ai/pricing): Free, Starter ($39/mo), Growth ($79/mo), Pro ($129/mo), Agency ($299/mo)

## Learning Resources

- [GEO Playbook](https://brandcited.ai/learn/geo-playbook): Complete guide to Generative Engine Optimization
- [AI Ranking Factors](https://brandcited.ai/learn/ai-ranking-factors): Every known AI citation signal
- [Schema Markup Guide](https://brandcited.ai/learn/schema-guide): Implementation guide with code examples

## Blog

- [Latest posts](https://brandcited.ai/blog): AI visibility insights and product updates

Step-by-step implementation#

Step 1: Create the file. Open a plain text editor. Name the file "llms.txt" (lowercase, no prefix). The file must be UTF-8 encoded plain text.

Step 2: Write the header. Start with your brand name and a one-line descriptor as an H1. Follow it with a 2-3 sentence description in blockquote format (> prefix).

Step 3: Add content sections. Create sections for your main content areas. Prioritize the content you most want AI engines to cite. Put your best content first in each section.

Step 4: Format links correctly. Each entry follows the pattern: - Title: Description. Keep descriptions under 15 words. Make them specific.

Step 5: Deploy the file. Upload llms.txt to your web root so it's accessible at yourdomain.com/llms.txt. Test by visiting that URL in a browser.

Step 6: Add to robots.txt. Add a reference in your robots.txt file to help crawlers discover it:

Sitemap: https://yourdomain.com/sitemap.xml LLMs-txt: https://yourdomain.com/llms.txt

Step 7: Verify. Run a BrandCited scan. The audit checks for llms.txt presence and validates its format.

Common mistakes to avoid#

Don't include every page. AI crawlers have limited attention budgets. Include your 20-50 most important pages, not your entire sitemap. Prioritize content that answers questions your target audience asks AI engines.

Don't use marketing language. The description should be factual, not promotional. "Leading AI visibility platform" is marketing. "AI visibility tracking across 9 platforms with site audit and growth actions" is descriptive.

Don't forget to update it. When you publish significant new content, add it to llms.txt. When you remove pages, remove the entries. Stale llms.txt files send AI engines to 404 pages.

Don't block the crawlers. Having an llms.txt file while blocking AI crawlers in robots.txt sends contradictory signals. Make sure your robots.txt and llms.txt are aligned.

Don't duplicate your sitemap. llms.txt serves a different purpose than sitemap.xml. The sitemap is comprehensive. llms.txt is curated. Only include pages you want AI engines to prioritize for citation.

Advanced: llms-full.txt#

Some sites also create an llms-full.txt file at their root. This is an extended version that includes more detail about each page: full descriptions, key topics covered, and target queries.

The format is the same as llms.txt but with longer descriptions and more entries. Use it when you want to give AI engines maximum context about your content.

Not all platforms read llms-full.txt. Start with llms.txt. Add the full version once your base implementation is verified and working.