// AI Crawler Audit

Which AI bots can actually read your site?

Paste a domain. We read its robots.txt and tell you which AI crawlers are allowed or blocked — and, when they're blocked, what it costs you in plain language.

Free · no login · one public robots.txt fetch, nothing else.

Why the training-vs-search distinction matters

Most sites that block AI crawlers do it by pasting a blocklist from an old blog post — and end up blocking the wrong ones. Blocking a training crawler (GPTBot, ClaudeBot, CCBot) is a deliberate content-licensing choice. Blocking a search / answer crawler (OAI-SearchBot, PerplexityBot, Claude-User) is almost always an accident — and it quietly removes you from the AI answers your customers are reading.

This tool separates the two so you can tell a choice from a mistake. Reads only the public /robots.txt; nothing is stored against your domain.