Robots.txt Generator
Generate a valid robots.txt file for your website with Allow, Disallow, Sitemap and Crawl-delay directives.
Advertisement
Generated robots.txt
Fill in the form and click generate to see your robots.txt here.
Robots.txt Generator
The Robots.txt Generator creates a valid robots.txt file that tells search engine crawlers which pages to crawl and which to skip. Use it to control bot access to your site, protect private directories and point crawlers to your sitemap.