This one may have just moved to a new location as I've been watching similar activity before which stopped. These new antics have been going on at this location for a week now and I waited just to make sure it was really coming from a common location which appears to be a block of IPs on some GoDaddy hosting farm secureserver.net.
This creepy crawler doesn't use any user agent string whatsoever and keeps asking for pages like "/#top" and other stupid stuff. Below is the range of IPs and the number of pages asked for just today. You'll note it was a slow day for them asking for only 75 pages, but the day isn't over yet.
18.104.22.168 [ip-68-178-242-111.ip.secureserver.net.] requested 30 pages as ""Performed an nslookup and got this:
22.214.171.124 [ip-68-178-242-126.ip.secureserver.net.] requested 15 pages as ""
126.96.36.199 [ip-68-178-242-128.ip.secureserver.net.] requested 15 pages as ""
188.8.131.52 [ip-68-178-242-127.ip.secureserver.net.] requested 15 pages as ""
nslookup 184.108.40.206When I did a whois on the IP there came the surprise:
220.127.116.11.in-addr.arpa name = ip-68-178-242-111.ip.secureserver.net.
[Querying whois.arin.net]Now do a whois on secureserver.net:
OrgName: Go Daddy Software, Inc.
Address: 14455 N Hayden Road
Address: Suite 226
178.128.0 - 178.255.255
NetRange: Registrant:Not sure it makes sense to block the entire GoDaddy IP range, so for now 18.104.22.168/24 is all I'm blocking unless I see more rogue activity in their network.
Special Domain Services, Inc.
14455 N Hayden Rd
Scottsdale, Arizona 85260
Registered through: WWDomains.com
Domain Name: SECURESERVER.NET
Created on: 30-Mar-98
Expires on: 29-Mar-12
Last Updated on: 07-Feb-06
BTW, anyone notice how many sneaky crawler networks I'm busting now that I have proximity alarms in place to spot organized activity?
This proximity alarm is great as it doesn't care if the crawlers ask for 1 page or 100 pages, the minute it detects multiple IP addresses in a similar range doing these things it pops up on my radar. The best thing is that the distributed crawler doesn't even have to use more than one IP address per day as long as they break one of my "bad bot rules" on each visit so the IP is flagged and archived. The proximity report of archived bad bot activity will then expose those archived bots operating from a single location.
Pretty tricky, eh?
You stupid bots better wise up quick, you can't hide behind a bank of IPs, your days are numbered!