robots.txt: no indexing Posted on 08/12/2013 by Ramoonus To disallow any bot from crawling your site, add the following to robots.txt robotUser-Agent: * Disallow: / Share this: Click to share on X (Opens in new window) X Click to share on Mastodon (Opens in new window) Mastodon Click to share on LinkedIn (Opens in new window) LinkedIn Click to share on Facebook (Opens in new window) Facebook