I saw this little Persaibot hit my site today without even looking at robots.txt and their website has the balls to say:
Persai uses this bot to crawl the web. It's probably the nicest bot with the greatest personality in the world. Seriously, give it some attention.Exactly how nice can a bot be that doesn't read robots.txt?
Did you read it and cache it some other day?
Doesn't matter, that was more than 24 hours ago, read it again.
I checked my logs from yesterday, it didn't read it then either and Persai hadn't visited my site in about a month before that.
I'm sorry, you have made the huge faux pas in robot rudeness.
Here's the intel I have on this little bot:
71.204.131.68 [c-71-204-131-68.hsd1.ca.comcast.net]Now the true irony here is that the CEO of Persai posted on his blog complaining about another search engine called Spock scraping every little bit of data about him but at least Spock claims to honor robots.txt.
"PersaiBot/2.1-dev3a (Persai web crawler; http://www.persai.com/bot.html; bot at persai dot com)"
67.202.55.205 [ec2-67-202-55-205.compute-1.amazonaws.com]
"Mozilla/5.0 (compatible; Persaibot/2.71828183; +http://www.persai.com/bot.html)"
76.102.193.127 [c-76-102-193-127.hsd1.ca.comcast.net]
"Mozilla/5.0 (compatible; Persaibot/2.71828183; +http://www.persai.com/bot.html)"
Must be a karma thing ;)
1 comment:
Amusingly, this sounds like the kind of hypocrisy that Ted Dziuba would bitch about on his blog.
Post a Comment