# robots.txt, www.nytimes.com 3/24/2005 # User-agent: * Disallow: /cgi-bin/