Saturday, November 18, 2006

Will Google Really Banish Scrapers?

Many people at PubCon, including some major companies, were telling me their tales of scraper horror. All the stories were similar about being endlessly abused and they were having trouble getting the problem under control or just gave up in frustration. Several people even asked the search engines what they were going to do about scrapers in the Q&A of some PubCon sessions and got the old "we're working on it" response which I think is half-hearted.

When you consider that AdSense technology fuels most scraper sites it's obvious Google could simply look at any AdSense account serving up ads from a multitude of locations which is usually a clue there's something rotten happening. Not that everyone with AdSense on multiple domains is bad, but when you see a single AdSense account used on thousands of locations, you know there's a good chance it's all crap. However, Google probably makes way too much money from scrapers just to eliminate them altogether. What's more than likely to happen is Google might drop scrapers from the Google index but leave their AdSense accounts intact so that the revenue stream continues from these sites being found in Yahoo and MSN.

Perhaps we can hope Yahoo and MSN figure out how to detect and eliminate scrapers first and put our friends at Google between a rock and a hard spot with the dilemma of scrapers vs. AdSense revenue. Either Google would have to clean up their search results to make the users happy or leave the scrapers in to make the stockholders and bean counters happy, which could backfire either way. Needless to say, I don't see scrapers going away any time soon because the financial incentives to keep them are just too great.

Meanwhile, I recommend reporting scrapers on Google's Report a Spam Result page and see if Google is serious about getting rid of scrapers when found.

No comments: