Can AI assistants read your site? Fixing a blocked robots.txt
Every day, WhoSaw checks whether ChatGPT, Perplexity and Claude can get into your site and read it.
If one is blocked, it's usually one line in a small file called robots.txt. The card in AI visibility shows the exact line to change.
Fix a blocked robots.txt
- Open AI visibility and press See the fix on the assistant marked Blocked.
- Look at In your robots.txt now. That's the line that keeps the assistant out.
- Press Copy the fixed lines.
- Under Where to change it, pick where your site is hosted: Cloudflare, Webflow, Wix, WordPress, or somewhere else. Follow its steps to paste the lines and save.
- Not doing it yourself? Press Email this fix to your web person, or Copy prompt for an AI coding tool.
- That's it. We check every day, so it usually shows as fixed by tomorrow. To see it sooner, press Check again.
What robots.txt is
A small file on your website that tells visiting bots which pages they may read. One wrong line can shut out an AI assistant without anyone noticing.
Each assistant has two visitors: one that finds your pages, and one that reads them when someone asks. Both need to get in.
"Likely blocked by your firewall"
This means we knocked on your site's door as the assistant would, and got a security check page instead of your site. It's most often Cloudflare's bot protection.
The card tells you which setting to change. We say likely because a firewall can treat the real assistant differently from our test.
Training data collectors
Some bots only collect text to train AI. Blocking them is your choice. It doesn't change whether AI assistants can read or recommend you today.
Check again
You can press Check again up to 3 times an hour for each website.
Watch how
Still stuck? Email support@whosaw.app