Monday, April 10, 2006

Yugoscrapia My Ass

The "Get a Fucking Clue" award for today goes to whoever the hell is sitting behind 212.102.136.25 hiding as Mozilla/5.0 (compatible; MSIE 5.0) from Yugoslavia that has spent over 8 hours now downloading 1,000+ error messages from my site and is currently still going strong.

Well pal, whoever you are, my automation is better than your automation because mine has a built-in BULLSHIT DETECTOR that stopped yours from doing it's job.

1 comment:

Anonymous said...

IncrediBILL,

I enjoy your posts your battles with the bots/scrapers/harvesters. The same vermin plague my sites as well. I use brute force to stop them for the moment but I'm considering writing a security module to shut them down as soon as they hit the site.

Have you considered creating (or know of any) a coop service/centralized Db where we could share known scraper IP addresses and UAs?

If each of the participants used one of the scripts that detected bot/scraper behaviour but then POST'ed the scum's IP address to a central server we could as a group shut most of these SOBs down.

If there were several hundred sites participating it wouldn't take long before we could weed out 80% of the assholes stealing our content.

-jay