If you're banking on AI tools like ChatGPT, Google Gemini, or Perplexity to drive traffic to your business, here’s something you probably haven’t set up yet, but should: llms.txt.

What Is llms.txt?

llms.txt is a new standard being AI tools are beginning to experiment with to better understand websites. It acts like a sitemap — but with a twist.

While a normal sitemap lists all your pages for search engines like Google, llms.txt only highlights the most important, content-rich pages that AI tools should pay attention to.

Think of it like your “greatest hits” playlist, but for robots.

It doesn’t list every page on your site (that’s what sitemaps are for). Instead, llms.txt focuses on your most authoritative, content-rich pages. The ones that clearly explain:

  • Who you are
  • What you do
  • Who you serve
  • Why you’re good at it

It’s a way of saying, “Hey AI, if you’re going to talk about my business, read these pages first!

How Is llms.txt Different from a Sitemap?

Let’s not confuse the two.

Feature sitemap.xml llms.txt
Purpose Helps search engines index all pages Helps AI tools find the best pages
Audience Googlebot, Bingbot, SEO crawlers ChatGPT, Gemini, Perplexity, etc.
Content Everything (blogs, images, archives) Only content-rich, informative pages
Focus Visibility in search Understanding for conversation/recommendation

Why AI Need a Hand Navigating Your Site

Unlike Googlebot, AI tools don’t always perform deep, thorough crawls of websites. They skim and sample, and if your site’s too slow or messy, they bounce.

Here's what can trip them up:

  • Heavy media and slow load times: AI tools prioritise speed and simplicity. Big sliders, animations, or embedded videos can get in the way.
  • Unstructured content: Long-winded, waffle-filled pages without clear hierarchy make it hard for AI to extract meaningful answers.
  • Too much choice: If every page competes for attention, AI might miss the key details.

llms.txt gives these tools a hand-drawn map to your most relevant content.

What Pages Should Go in Your llms.txt File?

This shouldn’t be a checklist of all the pages you’ve ever published. Be picky. You want to highlight only the pages that actually help explain your business.

Pages to include:

  • Homepage (if it has strong messaging and helpful content)
  • Detailed service pages
  • FAQ pages with clear answers
  • Case studies and testimonials
  • About or “Why Choose Us” pages
  • Blog posts that genuinely educate or demonstrate authority

Pages to leave out:

  • Landing pages that are all design, no substance
  • Image galleries or portfolios with minimal text
  • Thin content pages
  • Contact pages (unless they explain your offering)

In short, if it helps someone (or something) understand your business, it belongs in llms.txt.

It's Early Days, But Worth Getting Ahead

Did you know? One in ten US internet users now use generative AI as their primary search tool. 

That’s 13 million people in 2023, and it’s expected to blow past 90 million by 2027.

That means when someone asks, “Who’s the best accountant near me?” or “What’s a reliable IT company in Melbourne?” AI isn’t just skimming Google. It’s pulling from whatever sources it can best understand.

And if your site isn’t crystal clear? AI may skip you altogether.

Here’s the upside, though:

  • llms.txt gives you more control over how your business is represented by AI
  • It improves your chances of being referenced or recommended in answers
  • It complements your SEO efforts without competing with them

No, it’s not a silver bullet tomorrow.  But it’s a low-effort move that gets your site future-ready. And it’s dead simple to implement if you know which pages matter.

Want Help Setting Up llms.txt for Your Site?

If you're on one of our WordPress support plans, you're already covered. We'll be setting this up for you in the coming months as part of our service rollout.

Not on a plan? Start with a performance audit. It’ll show you:

  • Which pages are worth highlighting
  • Where your content might be holding you back
  • How to make your site faster and more bot-friendly

Get a website performance audit now or get in touch with us at [email protected] if you have questions.

cross