And I'm not talking about shaving my cock either, I'm talking about some scrapers running via bigpipeinc.com. From their web site it all looks like pro pipelines but I'd hate to whack an entire network just because of one business connection behaving badly.
Anyone else know anything about these guys?
Wednesday, May 10, 2006
Scraping My Big Pipe
Subscribe to:
Post Comments (Atom)
4 comments:
I've been blocking each offending host's class C.
I'm sure I have some collateral damage but I'm willing to suffer some friendly fire to stop the scraping bastards.
Depends on who the offending host happens to be as blocking a class C on AOL can fuck you up.
Hell, I turned up the heat on certain activity last week and snared a bunch of AOL IPs by accident, visibly cut into my revenue, and the amount of email from people that got caught skyrocketed.
Problem is hosts like AOL is they have shared IP's in a pool and people are allocated a new IP address about every 15 minutes or so which means a scraper beating the crap out of my site 10 minutes ago can be a sweet little old lady now online wanting to know why she's getting technobabble about spiders and not my web page.
Sometimes, you have to err on the side of caution ;)
[quote]Problem is hosts like AOL is they have shared IP's[/quote]
One person's problem is another person's opportunity :).
By the way, have you considered blocking stuff coming in with referrals from places like whois, or people doing searches on 'linkdomain', that kind of stuff?
I've considered such things, but at this time it's statistically insignificant based on other parameters that yeild better results.
The only referrals I block at this time are changing referrals which are always referral spam.
Post a Comment