I've taken a recent interest in tracking down the shitload crawlers of the domain sites out there that scrape your homepage, keywords, etc. and even display your AdSense and Google Analytics IDs.
Fucking asshats.
Posted a bunch of them on WebmasterWorld so drop in there to get details about domainsoutlook.com, statshow.com, urbandata.com, zitetrendz.com, hostnology.in, dawhois.com, clearwebstats.com, whoare.us, whoare.us, w3who.net, diigo.com, domainspyer.com, spyrush.com, aboutthedomain.com, seeallweb.org, webdetail.org and a minor update on domaintools.com.
Yes, I got busy :)
Wouldn't mind some feedback on my post about Spider Tracking Links - Examining 2 Methods - that would be nice!
Tuesday, November 08, 2011
Tracking Domain Intel Site Bots
Subscribe to:
Post Comments (Atom)
3 comments:
Hey Bill-- this was a pretty interesting article. What do you do with the info once you have tracked it? I would like to do some of that myself.
I block them from accessing my site obviously.
Certain things I can't control that they do, like gathering data from other 3rd party sites.
However, allowing them to directly mine my site, gather SEO data, and other things that I can control, NOT GONNA HAPPEN!
Blocked.
hope you still write here! and damn your good!
i found you through webmasterworld i always read your posts in user agent section.
glad there is someone out there to write about these things and speak up as 99% of others seem to be so damn naive as to how a proper bot should behave
Post a Comment