# /robots.txt for http://www.doe.mass.edu/ # comments to nmg@doe.mass.edu User-agent: EmailSiphon, ExtractorPro, Teleport, NICErsPRO, EmailCollector, CherryPickerSE/1.0, CherryPickerElite/1.0, EmailWolf 1.00, Crescent Internet ToolPak HTTP OLE Control v.1.0, EmailSiphon Disallow: / User-agent: * Disallow: /pdf Disallow: /img Disallow: /images Disallow: /mcas/search Disallow: /mcas/question.asp Disallow: /infoservices/research Disallow: /mcas/search Disallow: /frameworks/math/2000/all.asp Disallow: /WebResource.axd User-agent: * No-Email-Collection: http://www.doe.mass.edu/resources/policy.html?section=spam