Robots.txt Generator - Create a Robots.txt File for Your Site
Use this free robots.txt generator to create a valid robots.txt file that controls how search engine crawlers access your website. Specify allowed and disallowed paths, crawl delay, and your sitemap URL, then copy or download the result.
Generated robots.txt
User-agent: * Disallow: /admin/ Disallow: /private/ Sitemap: https://example.com/sitemap.xml
How does the Robots.txt Generator work?
This tool builds a robots.txt file following the Robots Exclusion Protocol. It combines your chosen user-agent, allow and disallow directives, an optional crawl-delay, and a sitemap reference into a single plain-text file that you place at the root of your domain (e.g. example.com/robots.txt) to guide search engine crawlers.