Two people asked us the same question during our bot traffic reality check live event, and we never got to either one live: does an llms.txt file actually stop AI from hammering your site?
We’ve sat with that question because the honest answer isn’t a quick yes or no. It’s that llms.txt was never built to do either of the jobs people hoped it would, and a lot of what’s been published about it this year has quietly led people to believe otherwise.
That confusion isn’t isolated. Across the questions we got, robots.txt, llms.txt, “AI crawlers,” and bot protection kept
Click here to continue reading this article.