# How to Add llms.txt for AI Search Optimization in Record Time

Learn how to add llms.txt & llms-full.txt to boost your website’s visibility in AI search engines like ChatGPT, Perplexity, and Deepseek. A step-by-step guide to structuring your content for AI indexing in under 60 minutes.

Mathias Ptacek•February 9, 2025

## How I Added llms.txt & llms-full.txt in No Time - And Why You Should Too

Rankscale is all about fast and agile improvements that sets it apart in the ever-changing AI landscape. Recently, I rolled out [llms.txt](/content/llms.txt) and [llms-full.txt](/content/llms-full.txt)!

Here’s a behind-the-scenes look at what they are, how they work, and a quick rundown of the implementation process inspired directly by the guidelines at llmstxt.org

## What’s the Deal with llms.txt?

**_llms.txt_** is a markdown-formatted text file that lives in your site’s root directory. Its job? Give AI a structured summary of your site’s content—without all the extra clutter. With **_llms.txt_**, you tell AI exactly what your site is about, ensuring it picks up the right information.

**Key Takeaway:** Traditional crawlers (like Google) are great at indexing websites, **but AI-based search engines and their simple crawlers often ignore JavaScript-rendered content**. That means sites built on React, Vue, Angular, or similar frameworks might not show up properly in AI-generated responses. So in LLMO terms, by using  **_llms.txt_** and **_llms-full.txt_** you can **boost your relevance in AI-driven search results dramatically.**

## **Why This Matters**

If you want AI-powered search engines and chatbots (like ChatGPT, Perplexity, [DeepSeek](/content/blog/deepseek_r1_revolutionizing_ai_with_transparent_reasoning_and_integrated_web_search/index.html) or Claude) to pull accurate information from your site, **_llms.txt_** is essential. **Here’s why**:

- **AI Search Engines Are on the Rise**  
AI search results are becoming more relevant in 2025 (e.g. [ChatGPT has just unlocked Web Search for free, nonregistered users](https://www.techradar.com/computing/artificial-intelligence/chatgpt-search-is-now-free-for-everyone-no-openai-account-required-is-it-time-to-ditch-google)) — and they work differently from traditional search engines. While Google indexes all pages, most LLMs don’t process JavaScript-rendered content. If your site relies on client-side rendering, it’s effectively invisible to AI search tools in the field of [Generative Engine Optimization (GEO)](/content/blog/explosive_facts_and_steps_to_master_ai_search_engine_optimization_in_2025/index.html).
- **Your Competitors are already optimizing for AI**  
Looking at the [directory.llmstxt.cloud](https://directory.llmstxt.cloud/), more and more companies are adapting to the standard 
- **Accurate Information = Better Visibility**  
Without a structured file, LLMs might misinterpret or completely miss your content. You don’t want AI chatbots making assumptions about your business, products, or services - as your customers already use AI answers heavily.
- **Faster AI Indexing:**  
With our new llms.txt files, AI systems can immediately pick up what our site is about. This means better visibility and more accurate search results for client-rendered content that might otherwise be overlooked.
- **It’s Fast, Easy, and Future-Proof**  
Creating **_llms.txt_** is ridiculously simple - it’s just a well-structured markdown file. But that small step future-proofs your site for AI-driven search and discovery. A few minutes of work, a couple of new files, and we’re ahead in the digital game!

## How to do it: Following the llms.txt Format

I based my implementation on the straightforward, effective format specified at llmstxt.org while looking at other implementations at directory.llmstxt.cloud

Here’s the step-by-step breakdown:

1. **Create a _llms.txt_ file in your root directory**  
   - Open a plain text editor (VS Code, Notepad++, or any IDE).  
   - Save the file as **`llms.txt`** in UTF-8 encoding.
2. **Follow the Recommended Format, here my Rankscale example (simplified):**  
   - Start with an **H1 header** ( _" **\# YourSiteName - optional tagline**"_).  
   - Add a **blockquote summary** (" _**\> A short description of your site**"_).  
   - Include **structured sections**( _" **\## YourSection**"_) with links to important content.

# Rankscale.ai - Monitor Your Website or SaaS or Brand or Service or Product or Client in AI Search Engines > Analyze and Optimize your AI Search Visibility with Results Tracking, Deep Insights and Actionable Recommendations - in Seconds. Here's what you can do with Rankscale:  
- Audit your website to see how AI understands it  
- Identify Gaps and Content Optimization Potential  
- Track your Presence in AI Search Results over Time  
- Analyze AI Result Citations & Brand Sentiments  
- Uncover, Survey, and Outpace Competitors  
- Rankscale works across: ChatGPT, AI Overviews, Perplexity, Claude, Gemini, Deepseek [Rankscale Landing Page](/content/site-root.html): Rankscale's landing page.

## Rankscale Features  
- [Features](/content/features.md): Learn about the features of Rankscale.  
## Why use Rankscale?  
- [Why Rankscale?](/content/whyrankscale.md): Learn why Rankscale is the best tool for AI search compared to traditional SEO tools.  
## Frequently Asked Questions  
- [Frequently Asked Questions](/content/faq.md): Answers to common questions about Rankscale, AI search, and Generative Engine Optimization (GEO).  
## Blog  
- [RankScale Blog](/content/blog/index.html): AI SEO, visibility tracking, and brand monitoring insights. Learn about AEO, GEO, and AI search optimization.

## Blog Posts  
- [AI SEO: Mastering AEO & GEO](/content/blog/explosive_facts_and_steps_to_master_ai_search_engine_optimization_in_2025/index.html): Master AI search engine optimization (AEO and GEO) in 2025: a guide.  
- [DeepSeek R1: Revolutionizing AI](/content/blog/deepseek_r1_revolutionizing_ai_with_transparent_reasoning_and_integrated_web_search/index.html): DeepSeek R1: Revolutionizing AI with Transparent Reasoning and Integrated Web Search

3. Add markdown files to extend/route to in your sections  
   - In _" **\## YourSection**"_, integrate a link (format: _" **[title](URL):description**"_) to the extended markdown content by using an additional " **_\*.md"** file --> See my " **_features.md_**" example for " **_\- [Features](/content/features.md): Learn about the features of Rankscale_**" below.

# Features  
## Website Audits  
Perform In-Depth AI Audits of Your Website  
Uncover how AI search engines perceive your content, pinpoint gaps, and discover optimal paths to greater visibility.  
- Content Scoring: Get detailed insights on relevance, authority, engagement, and technical structure that affect AI search visibility.
- Gap Analysis & Recommendations: Pinpoint which areas need improvement — with content recommendations on critical sections.
- Comprehensive Crawls: Select between single-page or multi-page crawling to fully reveal your site's AI readiness.
- Customizable Bot Settings: Simulate AI bots accurately by toggling anti-bot measures for realistic audits.

## Performance Tracking  
Monitor and Understand Your AI Presence  
See where and how often your brand appears in AI search results.  
- Specify your own search terms: Design custom AI search prompts that fit to your brand and industry - or leverage AI to propose them for you.
- Real-Time Updates: Analyze trends and changes to your brand's AI search rankings across time, topics and regions.
- Cross-Channel Insights: Get a unified view and compare results across different AI answer engines and industry topics.
- Sentiment Context: Understand how AI search engines perceive your brand and industry topics.

## Competitor Benchmarking  
Find and Track your Competitors  
Identify the key players in your AI search landscape, monitor their performance, and stay one step ahead.  
- Competitor Discovery: Automatically uncover direct and indirect competitors for relevant AI search terms and industry topics.
- Side-by-Side Comparisons: Compare visibility, sentiment, rankings and other important metrics to gain a holistic view of your competitors.
- Competitor Citations: See how often your competitors are cited and which sources are responsible for their visibility.
- Benchmarking: Identify Competitor's strengths and weaknesses across industry topics and AI engines.

4. **Upload the _llms.txt_ and " _\*.md"_ file(s) to your site's root directory**  
   - If using cPanel or FTP, place it at /public_html/llms.txt or /public/llms.txt  
   - If using Git, push it to your production build.  
   - This makes them **publicly accessible** to AI crawlers.  
   - Optionally you could create a path in your sitemap.xml as well (just a personal opinion)
5. **Verify accessibility**  
   - Open https://yourwebsite.com/llms.txt in your browser.  
   - Ensure it displays correctly and isn’t blocked by robots.txt.
6. **Optionally, add llms-full.txt** for detailed AI indexing  
   - Copy all structured markdown from your site into llms-full.txt.  
   - This provides AI engines a full content snapshot for deeper indexing.
7. **Update as needed**  
   - Just like **_robots.txt_**, keep it **fresh and accurate** as your site evolves.
   - Use tools like Mintlify to automate the creation and any updates

## **The Bottom Line**

If your website relies on JavaScript, or you simply want to ensure AI models interpret your content correctly, adding **_llms.txt_** or even **_llms-full.txt_** is a quick, no-brainer improvement. I did it in minutes (ok, ok, let's say <60 minutes - I'm a slow thinker), and now Rankscale's content is properly structured for AI.

## **Want your site to rank higher in AI search results?**
1. **Follow** this guide  
2. **Implement** _llms.txt_  
3. **Optimize and monitor** **your AI search visibility** with [Rankscale.ai](/content/site-root.html)

## Sources
- [1] [https://llmstxt.org/](https://llmstxt.org/) (A proposal to standardise on using an /llms.txt file)
- [2] [directory.llmstxt.cloud](https://directory.llmstxt.cloud/) (A curated directory of products and companies leading the adoption of the llms.txt standard)
- [3] [Techradar.com, 06.02.2025: ChatGPT Search is now free for everyone, no OpenAI account required – is it time to ditch Google?](https://www.techradar.com/computing/artificial-intelligence/chatgpt-search-is-now-free-for-everyone-no-openai-account-required-is-it-time-to-ditch-google)
- [4] [https://mintlify.com](https://mintlify.com/)
- [5] [https://rankscale.ai](/content/site-root.html)
- [6] [https://www.toolpilot.ai](https://www.toolpilot.ai/)