How To Shut Down Scrapers the AUP Way.

When I first started bot blocking the programmer in me saw it as a simple programming problem that could be solved with technology. Eventually, the realist in me saw that although I can solve a lot of individual webmaster abuse problems with technology that there's no way that a single bot blocking program can saturate a market deep enough to protect everyone.

Consider that bot block is literally putting a bandage on the problem on a server by server basis, or site by site basis, and not really solving the root of the problem for anyone.

Therefore, I've been looking for additional tools and methods to help everyone besides just the technological solutions I'm developing, and have been testing a real simple solution for effectively shutting down scrapers.

Without resorting to adding IP's to firewalls, .htaccess files, or even filing DMCA reports or any of that nonsense for copyright violation, I'm simply using the hosting company's AUP against the scrapers.

You'll find most hosting companies have the same boilerplate AUP clauses:

Unauthorized access to or use of data, systems or networks, including any attempt to probe, scan or test the vulnerability of a system or network or to breach security or authentication measures without express authorization of the owner of the system or network.

Along with this second gem of a clause:

Interference with service to any user, host or network including, without limitation, mailbombing, flooding, deliberate attempts to overload a system and broadcast attacks.

Well it doesn't take a rocket scientist to see that scraping fits into several categories mentioned above, so I wrote a few letters here and there to test the process and so far have been getting a near 100% success rate.

I would advise anyone catching a serious scraper in action to take a few minutes and send a simple AUP violation report to the hosting companies abuse department and include a log file of the violation.

See if that doesn't help eliminate some problems for everyone and not just bandage one server or site at a time.

Sure the scrapers can hop from ISP to ISP, but eventually nobody will take their business and they will have no place left to run. Maybe someone could even set up the equivalent of a SpamHaus for scrapers and their domains for easy reference, now that would be sweet.

Now if I could only find an automatic tool that sends scraper abuse notification reports at the end of the day.... sounds like I'll have to write it!

Try the AUP violation report approach and see if that works.

Please report back your success or failure, we want to know how it works for you!