# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / # A robot may not injure a human being or, through inaction, allow a human being to come to harm. # A robot must obey any orders given to it by human beings, except where such orders would conflict with the First Law. # A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.