Monday, November 19, 2007

FIRST LOOK: Yahoo Crawler Using Firefox UA

Woke up this morning to find my bot blocker had bitch slapped 300+ crawl attempts by Yahoo using the following criteria:

74.6.22.170 [llf520057.crawl.yahoo.net.] requested 302 pages as
"Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.4) Gecko/20071102 BonEcho/2.0.0.4"
Upon further examination it appears that this activity started on 11/17/2007 and the IP address used is a Yahoo proxy and some of the forwarded IPs were:
74.6.18.46 -> rz502516.crawl.yahoo.net.
74.6.18.160 -> rz502426.crawl.yahoo.net.
74.6.18.163 -> rz502429.crawl.yahoo.ne

a lot more 74.6.18.* IPs etc., you get the idea...
What was curious is the version of Firefox claimed to be Bon Echo which if I'm not mistaken was pre-release Firefox 2 code.

Didn't look like they were making screen shots based on todays activity unless they had already cached the images so I'm not sure what in the hell Yahoo's up to at this point.

Take a look in your logs as I find it hard to believe I'm the only one seeing this.

2 comments:

Anonymous said...

They're over here as well. Haven't looked at the amount of requests but saw them in the logs.

Anonymous said...

They've been doing some weird stealth action on our sites as well. Recently caught a Yahoo Crawler with libwww-perl as the UA. Google has been doing the same thing...caught Google AdWords bot posing as Internet Explorer.