Thursday, June 15, 2006

Why is MonsterCommerce mining my site?

Didn't think this was worth mentioning until it happened several days in a row.

This is what's requesting various web pages:
"Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)"
So the question is what do they want?

Are they scraping my directory for potential customer leads?

Very curious indeed.

1 comment:

MP1 said...

The one that hit your site is a lot of scrapes come from that netrange
The one that hit my site came from Hosting-Network GmbH. Their CIDRs are: and and

Each one of these gets a "Duplicate zone found ( is repeated)" error on reverse dns. If it comes from any of the Hosting-Network GmbH ip addresses, it's banned along with ranges too.
If I'm not mistaken, Hosting-Network GmbH is German.