Posted inWebdesign robots.txt: no indexing by Ramoonus08/12/201303/01/2016 To disallow any bot from crawling your site, add the following to robots.txt robotUser-Agent: * Disallow: /