In the badly behaving corporate bots dept. we offer Netsweeper as our newest entry from Canada. They run one of those content filtering companies that thinks they should be allowed to crawl your site no matter what just to protect their clients.
Sorry, but we happen to disagree with all these content filtering spiders that feel the need to crawl without any regard for robots.txt and we really don't need a whole buttload of content filtering companies scanning the fucking web.
Yes, I threw in the word fucking just so your asshole spider will flag this post as bad content so none of your goddamn customers can read this so blow that out your ass.
Let's see what Netsweeper runs:
66.207.120.226 "webcollage/1.127"These IP addresses have the following host names:
66.207.120.226 "NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)"
66.207.120.227 Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
66.207.120.226 -> firewall.net-sweeper.comLet's just cut thru the chase and here's the information to block their ass:
66.207.120.227 -> host227.net-sweeper.com
CustName: NetsweeperTa ta Netsweeper, you've been blocked and swept under my rug.
Address: 4-512 Woolwich Street
City: Guelph
StateProv: ON
PostalCode: N1H-3X7
Country: CA
RegDate: 2003-04-08
Updated: 2003-04-08
NetRange: 66.207.120.224 - 66.207.120.239
Cya!
6 comments:
They've got another small block which you probably like to nuke on that occasion, too:
216.171.98.64/26
Arin's whois server comes in very handy for gathering netblocks of a particular corporation. Just run on the Unix shell of your choice:
whois -h whois.arin.net "Netsweeper*"
And it will return a collapsed list of any entry it can get hold of containing "Netsweeper". Keep in mind that the output is limited to 255 entries, so the query should be sufficiently specific in order to be useful.
Thanks, but I know how to look 'em up, sometimes just get lazy or in a rush to go out to lunch in today's case ;)
Fine, let's do it thoroughly...
Netsweeper Inc. NETSWEEPER-ATRIA-1 (NET-216-171-98-64-1) 216.171.98.64 - 216.171.98.127
Netsweeper FW-NETSWEEPER-1 (NET-66-207-120-224-1) 66.207.120.224 - 66.207.120.239
Netsweeper FW-NETSWEEPER-2 (NET-66-207-119-232-1) 66.207.119.232 - 66.207.119.239
Nevermind,
just thought it might be useful for some of your readers who haven't heard of the features unique to ARIN yet
Btw, the 2nd line is already included in the follow up, thus a bit redundant ;-)
Oh dear,
s/2nd line/3rd line/g
It's getting late in Central Europe ;-)
I know how to look them up too, but there's still a lot I don't know about blocking misbehaving bots. Have you ever done a basic tutorial, or would you consider doing one?
Post a Comment