ServePath to being banned

Found a bunch of random stuff coming from a hosting company called ServePath today while running historical analysis on a batch of IPs.

Now these are the visible crawlers that came from ServePath:

64.151.75.252 PEAR HTTP_Request class ( http://pear.php.net/ )
64.151.64.212 "Jakarta Commons-HttpClient/3.0"
64.151.65.12 "Jakarta Commons-HttpClient/3.0"
64.151.111.116 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
64.151.112.44 NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)

Here's the whole range:

OrgName: ServePath, LLC
NetRange: 64.151.64.0 - 64.151.127.255
CIDR: 64.151.64.0/18

I'm going to block the whole thing and see if there are any stealth crawlers operating out of that location that haven't tripped any alarms yet and see what happens.

ServePath to more IPs

Just after I blocked the last batch of IPs from ServePath someone popped up on yet a new location ranging from 69.59.128.0 to 69.59.191.255 . All of their reverse DNS starts with "customer-reverse-entry." so I think I'll just zap them by that host address phrase and save some trouble here.

Discovery Engine's Discobot Discovered My Bot Blocker

I found this little Discobot from Discovery Engine trying to dance around on my server but the bot blocker bouncer at the door was already keeping him behind the velvet ropes.

Here's a sample of what I saw on my site:

208.96.54.74 "GET /robots.txt"
"Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)"

208.96.54.68
"Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)"

It does honor robots.txt just like they said it did but it cached it for about 48 hours between visits.

They were nice enough to provide the range of IPs it uses:

208.96.54.67 - 208.96.54.96

Those IPs are from Servepath which I already block.

Between whitelisting allowed bots and blocking more data centers then I'd care to admit, this poor little Discobot didn't stand a chance to discover anything.

Call back when you're all grown up and ready to send traffic.