Sunday, October 15, 2006

Netsweeper Caught Using Multiple Brooms

In the badly behaving corporate bots dept. we offer Netsweeper as our newest entry from Canada. They run one of those content filtering companies that thinks they should be allowed to crawl your site no matter what just to protect their clients.

Sorry, but we happen to disagree with all these content filtering spiders that feel the need to crawl without any regard for robots.txt and we really don't need a whole buttload of content filtering companies scanning the fucking web.

Yes, I threw in the word fucking just so your asshole spider will flag this post as bad content so none of your goddamn customers can read this so blow that out your ass.

Let's see what Netsweeper runs: "webcollage/1.127" "NutchCVS/0.7.2 (Nutch;;" Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
These IP addresses have the following host names: -> ->
Let's just cut thru the chase and here's the information to block their ass:
CustName: Netsweeper
Address: 4-512 Woolwich Street
City: Guelph
StateProv: ON
PostalCode: N1H-3X7
Country: CA
RegDate: 2003-04-08
Updated: 2003-04-08

NetRange: -
Ta ta Netsweeper, you've been blocked and swept under my rug.



Olliver said...

They've got another small block which you probably like to nuke on that occasion, too:

Arin's whois server comes in very handy for gathering netblocks of a particular corporation. Just run on the Unix shell of your choice:

whois -h "Netsweeper*"

And it will return a collapsed list of any entry it can get hold of containing "Netsweeper". Keep in mind that the output is limited to 255 entries, so the query should be sufficiently specific in order to be useful.

IncrediBILL said...

Thanks, but I know how to look 'em up, sometimes just get lazy or in a rush to go out to lunch in today's case ;)

IncrediBILL said...

Fine, let's do it thoroughly...

Netsweeper Inc. NETSWEEPER-ATRIA-1 (NET-216-171-98-64-1) -
Netsweeper FW-NETSWEEPER-1 (NET-66-207-120-224-1) -
Netsweeper FW-NETSWEEPER-2 (NET-66-207-119-232-1) -

Olliver said...

just thought it might be useful for some of your readers who haven't heard of the features unique to ARIN yet

Btw, the 2nd line is already included in the follow up, thus a bit redundant ;-)

Olliver said...

Oh dear,

s/2nd line/3rd line/g

It's getting late in Central Europe ;-)

Uke Xensen said...

I know how to look them up too, but there's still a lot I don't know about blocking misbehaving bots. Have you ever done a basic tutorial, or would you consider doing one?