LLMs.txt Validator
Validate your llms.txt and llms-full.txt files against the proposed standard. Ensure your site is perfectly optimized for AI agents and LLMs.
Ensure your website is perfectly optimized for AI agents and Language Models. Validate your llms.txt and llms-full.txt files for proper structure, compliance, and functionality.
What is LLMs.txt?
llms.txt is a revolutionary proposed file standard that helps Large Language Models (LLMs) understand your website's structure and find your most important content efficiently.
Quick Definition
/llms.txt is a Markdown file placed at your website's root directory that provides AI systems with a curated, structured overview of your site's most valuable content.
Think of it as: A navigation guide specifically designed for AI - just like robots.txt guides web crawlers, but llms.txt guides intelligent AI systems.
AI-First Approach
Optimized specifically for LLMs and AI agents, not traditional search engines.
Context Window Friendly
Clean, curated content that fits within LLM context limitations.
Web Standard
Part of the emerging GEO (Generative Engine Optimization) movement.
The Problem
- HTML pages are cluttered with ads & navigation, confusing AI.
- Dynamic content heavily reduces AI reading accuracy.
- Important content gets lost in unnecessary background noise.
- Context window limitations force AI to ignore entire pages.
The Solution
- Clean, curated content summaries that AI easily reads.
- Structured Markdown format with a focus on core information.
- Direct, easily identifiable links to key documentation pages.
- AI-optimized organization to guarantee data ingestion.
Understanding LLMs-Full.txt
llms-full.txt is the companion file to llms.txt that contains complete documentation content rather than just structured links.
What Makes It Different?
While /llms.txt provides a structured index with links, /llms-full.txt contains the actual content - full documentation, guides, and reference material in one comprehensive file.
llms.txt (Index)
- Structured format (H1, blockquote, H2 sections)
- Links to important pages with descriptions
- Small file size (<50KB typically)
- Quick overview for AI navigation
llms-full.txt (Content)
- Free-form markdown content
- Complete documentation in one file
- Large file size (can be >500KB)
- Self-contained reference material
✅ Great For
- Comprehensive API documentation
- Complete user guides and tutorials
- Technical reference materials
- AI systems with large context windows
- Self-contained documentation sites
⚠️ Consider Carefully
- Files larger than 1MB (may hit limits)
- Frequently changing content
- Sites with many separate topics
- Limited server bandwidth
- Mobile or low-power AI systems
How Validation Differs
llms.txt Validation
- • Requires H1 header
- • Requires blockquote summary
- • Requires H2 sections with links
- • Strict structural rules
llms-full.txt Validation
- • Flexible content structure
- • Valid markdown format
- • File size warnings (>500KB)
- • Focus on readability
Note: Our validator currently applies llms.txt rules to both file types. This will be updated to use appropriate validation for each format soon.
Understanding LLMs.txt: The New Web Standard
Learn how LLMs.txt compares to existing web standards and traditional SEO.
Similar to Other Standards
SEO vs GEO
Traditional SEO
Optimization for search engines like Google and Bing. Focuses heavily on keywords, backlinks, UI/UX, and page authority.
New GEO (Generative Engine Optimization)
Optimization specifically for LLMs and AI agents, where llms.txt is a core tool. Focuses on clear structure, comprehensive answers, and machine readability.
Why Proper LLMs.txt Validation Matters
Automated checking helps you easily match web standards and quickly identify parsing issues.
Required Structure
- H1 HeaderPrimary site or project name
- Blockquote summaryConcise introduction for LLMs
- Section HeadersAt least one H2 section with links
Common Issues We Catch
- Broken or malformed URLs
- Duplicate URL entries
- Missing required sections
- Incorrect Markdown formatting
- Sizes too large (>500KB)
Best Practices
- File PlacementAccessible directly at /llms.txt
- Current ContentAll links must be fully valid
- Size ManagementSmall size for token limits
Common Mistakes vs Best Practices
Mistakes to Avoid
Missing Elements
No H1, missing blockquote summary, or no H2 sections.
URL & Link Issues
Broken URLs, missing https:// prefixes, or duplicate links.
Formatting Bugs
Incorrect Markdown levels or malformed list layouts.
Best Practices
Content Structure
Start with a clear H1, write a concise summary, and group related links.
Link Selection
Include key documentation and API references. Skip marketing or login pages.
File Placement
Ensure the file is always directly accessible at /llms.txt.
Keys to LLMs.txt Success
Be Structured
- Always use a clear H1
- Include a blockquote summary
- Group links under H2 sections
Be Accurate
- Use fully-qualified URLs
- Provide descriptive titles
- Remove duplicate links
Be Concise
- Keep file sizes under 50KB
- Link to essential content
- Avoid marketing fluff
Why Use an LLMs.txt Validator?
Prevent AI Errors
Malformed files can cause AI systems to misinterpret your content or miss important pages.
Improve AI Discovery
Properly structured files help AI systems find and understand your most valuable content.
Save Time
Catch errors instantly instead of debugging why AI can't understand your site.
Key Validation Checks
Structure & Syntax
Ensures proper Markdown headings, blockquotes, and lists for parsers.
URL Validation
Checks that all links are fully-qualified and lead to actual content.
Metadata Completeness
Verifies that project descriptions and optional fields are correctly formatted.
File Size Limits
Calculates token usage to ensure it easily fits in modern context windows.
Common Questions about LLMs.txt
What is llms.txt and why is it important?
llms.txt is a proposed file placed at the root of a website (e.g., /llms.txt) in Markdown format to guide LLMs through the site's most important content in a clear, structured way. It exists because HTML pages are often cluttered.
How is llms.txt similar to other web standards?
robots.txt: controls web crawlers
sitemap.xml: lists site pages
llms.txt: explicitly signals to LLMs where and how to fetch content
What's the difference between SEO and GEO?
SEO optimizes for search engines like Google and Bing. GEO optimizes for LLMs and AI agents. llms.txt is a core tool for Generative Engine Optimization.
Who is already using llms.txt?
Major projects: nbdev, FastHTML, LangGraph, LangChain, Fast.ai
Commercial: Mintlify, Anthropic, Cloudflare, and Yoast
What's the difference between llms.txt and llms-full.txt?
llms.txt contains structured links to your important pages. llms-full.txt includes the full plain-text content of documentation for direct ingestion.
Is this validator free to use?
Yes! Our llms.txt validator is completely free to use. No registration required, no limits on usage. We're committed to helping the web become more AI-friendly.