Back in November I wrote an open letter to the Amazon AWS Group about trying to get them to stop using the default user agent "Java/1.5.0_09".
Today I noticed that they gave me a clear response to my open request:
18.104.22.168 [domU-12-31-33-00-02-01.usma1.compute.amazonaws.com.]Oh yes, prefixing "Java/1.5.0_09" with an MSIE 6.0 user agent is MUCH better.... NOT!
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461) Java/1.5.0_09"
Must've been getting blocked from crawling too many sites that block the default Java UA.
Nice try guys, but that's really fucking lame.