What the result means
robots.txt is not access control and cannot prove a requester's identity. Protect sensitive content with authentication, authorization, and server-side controls.
AI Crawler robots.txt Checker
Check robots.txt rules locally for OAI-SearchBot, GPTBot, ClaudeBot, Claude-SearchBot, and PerplexityBot on a path you specify.
No upload
Paste robots.txt and enter a path or URL to check OAI-SearchBot, GPTBot, ClaudeBot, Claude-SearchBot, and PerplexityBot. Content is processed locally in this browser and the page never fetches the target site.
Check results
Add content and select Analyze rules to see results.
How to check AI crawler rules
- Paste robots.txt
- Enter the path to check
- Review each crawler's matched group and effective rule
Local privacy mode
Inputs and analysis stay on this page. Nothing is written to the URL, browser storage, logs, or analytics.