I've taken a recent interest in tracking down the shitload crawlers of the domain sites out there that scrape your homepage, keywords, etc. and even display your AdSense and Google Analytics IDs.
Fucking asshats.
Posted a bunch of them on WebmasterWorld so drop in there to get details about domainsoutlook.com, statshow.com, urbandata.com, zitetrendz.com, hostnology.in, dawhois.com, clearwebstats.com, whoare.us, whoare.us, w3who.net, diigo.com, domainspyer.com, spyrush.com, aboutthedomain.com, seeallweb.org, webdetail.org and a minor update on domaintools.com.
Yes, I got busy :)
Wouldn't mind some feedback on my post about Spider Tracking Links - Examining 2 Methods - that would be nice!
Tuesday, November 08, 2011
Tracking Domain Intel Site Bots
Subscribe to:
Post Comments (Atom)
2 comments:
Hey Bill-- this was a pretty interesting article. What do you do with the info once you have tracked it? I would like to do some of that myself.
I block them from accessing my site obviously.
Certain things I can't control that they do, like gathering data from other 3rd party sites.
However, allowing them to directly mine my site, gather SEO data, and other things that I can control, NOT GONNA HAPPEN!
Blocked.
Post a Comment