# robots.txt to disallow overload by evil robots User-agent: * Disallow: /cgi-bin/ User-agent: wget Disallow: / User-agent: Twiceler Disallow: / User-agent: Twiceler-0.9 Disallow: /