# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: User-Agent: * # Disallow: / Disallow: /socioec/blocks/ Disallow: /socioec/blockgroups/ Disallow: /socioec/places/ Disallow: /socioec/tracts/ Disallow: /socioec/counties/ Disallow: /socioec/states/ Disallow: /lep/blocks/ Disallow: /lep/blockgroups/ Disallow: /lep/places/ Disallow: /lep/tracts/ Disallow: /lep/counties/ Disallow: /lep/states/