OK, everyone knows I'm a big Nutch fan.
Everyone should also know the above statement is sarcasm laid on thick like peanut butter.
Anyway, back to the topic...
I saw this supposedly Nutch user agent in my logs today which is pretty slimy claiming to be MSIE, Opera, and Nutch all in the same UA.
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.9 (http://lucene.apache.org/nutch/about.html; http://lucene.apache.org/nutch/bot.html; mail@dev.null)Whoever thought that was clever enough to slide under my radar was wrong as that word Nutch set off alarms and tossed your crawl into the online equivalent of the pit of despair.
Sorry, better luck next time.
2 comments:
Correct me if I'm wrong, but isn't Nutch an open source search engine still under development, not a web browser?
Yes, you are correct.
That's why that particular user agent was interesting enough to post.
Post a Comment