To disallow any bot from crawling your site, add the following to robots.txt

robotUser-Agent: *
Disallow: /

Leave a comment

Your email address will not be published. Required fields are marked *