LLMs.txt
LLMs.txt is an emerging web standard that allows website owners to provide AI systems and large language models with structured information about site content, usage permissions, preferred citations, and training data preferences. Analogous to robots.txt for traditional crawlers, it gives publishers a standardized mechanism to communicate with AI systems about how their content should be used, attributed, and retrieved.
Why It Matters
As AI systems increasingly train on and retrieve web content, publishers need clear channels to communicate permissions and preferences. LLMs.txt provides a proactive, structured way to signal AI-friendliness, specify attribution requirements, and guide AI systems toward the most authoritative and current content on your site.
How It Works
An LLMs.txt file is placed at the root of a domain and written in Markdown format, providing a human and machine-readable overview of the site's content, structure, and AI usage guidelines. It can include links to key content sections, API endpoints, schema resources, and specific usage terms for AI training and retrieval.
Use Cases
- Publishers specifying which content sections can be used for AI training
- B2B companies pointing AI retrieval systems to their most authoritative documentation
- News organizations communicating real-time content update locations to AI systems
- E-commerce sites directing AI shopping agents to structured product data feeds
- Research institutions specifying attribution requirements for AI-generated content citing their work
Best Practices
- Implement LLMs.txt alongside robots.txt-they serve different but complementary functions
- Include links to your most important, AI-retrievable content in the file
- Specify clear attribution preferences if your content is used in AI training or retrieval
- Update LLMs.txt whenever major site structure or content permission changes occur
- Follow the evolving standard at llmstxt.org to stay current with specification updates
- Use LLMs.txt as a signal of AI-readiness and transparency to build AI system trust
Frequently Asked Questions
Is LLMs.txt an official standard or still experimental? +
What is LLMs.txt? +
Why does LLMs.txt matter? +
Related Terms
Control how LLMs are allowed to use your content
Use llms.txt to signal how AI systems may use your content and track your AI visibility as you refine your policy.
No credit card required • Start in minutes