Thursday, June 15, 2006

Why is MonsterCommerce mining my site?

Didn't think this was worth mentioning until it happened several days in a row.

This is what's requesting various web pages:

72.32.59.114
wordsmith.monstercommerce.com
"Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)"
So the question is what do they want?

Are they scraping my directory for potential customer leads?

Very curious indeed.

1 comment:

MP1 said...

The one that hit your site is rackspace.com a lot of scrapes come from that netrange
The one that hit my site came from Hosting-Network GmbH. Their CIDRs are:
64.17.128.0/18 and 208.44.62.0/27 and 71.18.0.0/16.

Each one of these gets a "Duplicate zone found (128.17.64.in-addr.arpa. is repeated)" error on reverse dns. If it comes from any of the Hosting-Network GmbH ip addresses, it's banned along with RackSpace.com ranges too.
If I'm not mistaken, Hosting-Network GmbH is German.