radar
lock Free · Private · In-Browser
Robots.txt Tester
Paste robots.txt rules, enter a path, and see whether a crawler should be allowed or blocked by the longest matching rule.
-
Check crawler rules before publishing
A small robots.txt mistake can hide important pages from search engines or expose staging areas you meant to block.
This tester helps you reason about common robots directives without touching the live file.
Frequently Asked Questions
Does this fetch my live robots.txt?
No. Paste the rules you want to test. This keeps the tool browser-only and avoids network calls.
How is the result calculated?
It checks rules for the selected user-agent and wildcard sections, then uses the longest matching Allow or Disallow path.
Is this a full search engine simulator?
No. It is a practical tester for common Allow and Disallow rules.
Take it further
SEO & Web Design Services
Need metadata, content structure, and technical SEO cleaned up across a full site? I can help.