User-agent:HotSheet.com Disallow:/ User-agent:gaykelowna.com Disallow:/ User-agent:gayokangan.com Disallow:/ User-agent:Googlebot User-agent:slurp User-agent:msnbot User-agent:teoma User-agent:ask.com User-agent:AskJeeves User-agent:Atomz.com User-agent:infoseek.com User-agent:lycos User-agent:netmechanic User-agent:Inktomi Slurp User-agent:Smart Spider User-agent:TechBot User-agent:W3M2 User-agent:WebQuest User-agent:Digger User-agent:www.diggit.com User-agent:WDG_SiteValidator Disallow: /cgi-bin/ Disallow: /images/ Disallow: /style/ User-agent: Mediapartners-Google* Disallow: / User-agent:* Disallow:/cgi-bin/ #User-agent wildcard (*) allows all spiders. #Disallow field empty allows all files to be retrieved. #Disallow /cgi-bin/ doesnt allow into cgi-bin. #http://www.searchengineworld.com/robots/robots_tutorial.htm #http://www.thesitewizard.com/archive/robotstxt.shtml