
If you're banking on AI tools like ChatGPT, Google Gemini, or Perplexity to drive traffic to your business, here’s something you probably haven’t set up yet, but should: llms.txt.
llms.txt is a new standard being AI tools are beginning to experiment with to better understand websites. It acts like a sitemap — but with a twist.
While a normal sitemap lists all your pages for search engines like Google, llms.txt only highlights the most important, content-rich pages that AI tools should pay attention to.
Think of it like your “greatest hits” playlist, but for robots.
It doesn’t list every page on your site (that’s what sitemaps are for). Instead, llms.txt focuses on your most authoritative, content-rich pages. The ones that clearly explain:
It’s a way of saying, “Hey AI, if you’re going to talk about my business, read these pages first!”
Let’s not confuse the two.
| Feature | sitemap.xml | llms.txt |
| Purpose | Helps search engines index all pages | Helps AI tools find the best pages |
| Audience | Googlebot, Bingbot, SEO crawlers | ChatGPT, Gemini, Perplexity, etc. |
| Content | Everything (blogs, images, archives) | Only content-rich, informative pages |
| Focus | Visibility in search | Understanding for conversation/recommendation |
Unlike Googlebot, AI tools don’t always perform deep, thorough crawls of websites. They skim and sample, and if your site’s too slow or messy, they bounce.
Here's what can trip them up:
llms.txt gives these tools a hand-drawn map to your most relevant content.
This shouldn’t be a checklist of all the pages you’ve ever published. Be picky. You want to highlight only the pages that actually help explain your business.
Pages to include:
Pages to leave out:
In short, if it helps someone (or something) understand your business, it belongs in llms.txt.
Did you know? One in ten US internet users now use generative AI as their primary search tool.
That’s 13 million people in 2023, and it’s expected to blow past 90 million by 2027.
That means when someone asks, “Who’s the best accountant near me?” or “What’s a reliable IT company in Melbourne?” AI isn’t just skimming Google. It’s pulling from whatever sources it can best understand.
And if your site isn’t crystal clear? AI may skip you altogether.
Here’s the upside, though:
No, it’s not a silver bullet tomorrow. But it’s a low-effort move that gets your site future-ready. And it’s dead simple to implement if you know which pages matter.
If you're on one of our WordPress support plans, you're already covered. We'll be setting this up for you in the coming months as part of our service rollout.
Not on a plan? Start with a performance audit. It’ll show you:
Get a website performance audit now or get in touch with us at [email protected] if you have questions.