The other day I posted about how VT.EDU's tiki was overflowing with spam so today I went thru my spam filter log to just see how many attempted spams there were last week using tiki redirect pages.
Here's a short list of the most recent attempted spams linking to tikis that hit my server:http://www.lug-viersen.de/tiki-directory_redirect.php?siteId=136#viagra
What's distressing is that Google and the other SE's really love these spammed pages too, just gobble them up, and it's probably unwittingly passing PR from all these spammed tiki sites on such terms as viagra, cialis, levitra and a whole lot more.
http://ipvs.informatik.uni-stuttgart.de/BV/swarmrobot/tikiwiki-1.9.2/tiki-directory_redirect.php?siteId=474#viagra
http://i60p4.ira.uka.de/tiki/tiki-directory_redirect.php?siteId=24#viagra
http://www.xsl-rp.de/tiki-directory_redirect.php?siteId=1018#cialis
http://www.neurotransmitter.net/wiki/tiki-directory_redirect.php?siteId=243#viagra
http://research.cs.vt.edu/advance/tiki/tiki-directory_redirect.php?siteId=3284#viagra
http://meverhagen.nl/tikiwiki/tiki-directory_redirect.php?siteId=19#viagra
http://www.namurantifasciste.be/tiki-directory_redirect.php?siteId=996#viagra
http://www.ee.aston.ac.uk/intranet/tiki-directory_redirect.php?siteId=10#viagra
http://www.xsl-rp.de/tiki-directory_redirect.php?siteId=1015#viagra
http://www.railfuture.org.uk/tiki-directory_redirect.php?siteId=61#viagra
http://www.ee.aston.ac.uk/intranet/tiki-directory_redirect.php?siteId=9#viagra
http://herenaforge.org/tiki-directory_redirect.php?siteId=38#phentermine
http://herenaforge.org/tiki-directory_redirect.php?siteId=51#viagra
http://www.derrychineseschool.org/DCS/tiki-directory_redirect.php?siteId=7#viagra
http://openg.org/tiki/tiki-directory_redirect.php?siteId=54#viagra
http://www.prospace.org/tiki-directory_redirect.php?siteId=2385#viagra
http://www.milwaukeelug.org/tiki/tiki-directory_redirect.php?siteId=1349#viagra
http://www.ee.aston.ac.uk/intranet/tiki-directory_redirect.php?siteId=18#viagra
http://dev.librehwdb.tuxfamily.org/tiki-directory_redirect.php?siteId=18#viagra
So Google gives spammers a 2-for-1 special by giving them SEO value for their spamming activities, it's just a crying shame, it really is.
What's pathetic is this problem could be stopped on both sides of the coin. The tiki/wiki software developers could get off their lazy asses and implement some tools to allow webmasters to stop this rampant spamming of their software, it's easily doable. Additionally, the search engines like Google can easily identify and stop indexing spammed web pages to eliminate the value they give to the spammer.
Remember, I'm reporting about ATTEMPTED spams, all those links and a shitload more were automatically dumped, it's not rocket science, it's barely programming above a rudimentary level to identify and filter that shit out.
Why does this continue when the solutions are so simple for all involved?
Amazing that it's allowed to continue, simply amazing.
Saturday, October 07, 2006
Vulnerable Tikis Ruthlessly Spammed and Google Indexed
Thursday, October 05, 2006
Podomatic Vulnerability Enables Spammer Redirects
Here's another instance in a rash of reported vulnerabilities in member registration pages being spammed. Never heard of Podomatic before but it appears the spammers sure have and some nitwit registered as a member called Valium to do his spamming.
The link to the member's site is:
http://www.podomatic.com/profile/member/valiumThe javascript redirect code appears to be this shit embedded in the memberpage:
<script>Just goes to show you that if you don't secure your sites some spammer will abuse it but people just don't listen.
var mbht872 = 'on=';
var bikmr354 = 'qiqyi199';
var zlh171 ='ment';
var k97='.lo';
var ydxglyjedai737='ti';
var bmmp211='docu';
var mzcra833='http://drsearch.net/search.php?aff=15313&q=';
var ertmj632='valium';
var qiqyi199 = 'ca';
var lflx482='"';
if(bikmr354 = 'qiqyi199')eval(bmmp211+zlh171+k97+qiqyi199+ydxglyjedai737+mbht872+lflx482+mzcra833+ertmj632+lflx482);
</script>
Wednesday, October 04, 2006
Automatic Detection of Spam Hand Jobs
Sometimes certain anti-spam ideas just hit you upside the head when you least expect them and seem so obvious you wonder what took you so long to figure it out.
I've already blogged about the fact that I've stopped all automated spam dead in it's tracks on my sites, but people manually posting can of course correct all of the errors detected and continue to make an unwanted garbage post.
I have an extensive junk detection filter that rejects anything with the usual suspects like viagra, cialis, gambling, poker, etc. which stops the nastiest of these posts. However, some little pain in the ass SEO aka spammer might slip thru with a hand job posting about his store in India selling magic beetle dung or something that you would never imagine putting in your junk filter in the first place.
A few days ago I decided to review the last 30 days of legitimate submissions and compare them to the few off topic hand jobs that slipped through the cracks and see if I could come up with anything that would allow me to stop the hand jobs of absolutely random and crazy things outside the realm of the typical common auto-spam posts.
Then, like a lightning bolt it suddently hit me, that with these random off topic hand spams it's not what's IN the posts it's what's NOT in the posts that makes them easily identifiable. The concept is to scan for a list of words that SHOULD be in the post, like quotes from anything in the thread or certain keywords related to the topic and automatically set everything to MODERATE that doesn't fit the usual posting patterns.
Basically it's a 'lack of content filtering' technique and off topic posts, like spam, stand out like a sore thumb.
Using this blog as as example for a topic, you would expect most comments to contain words like bot, spam, IP, host, crawl, firewall, htaccess, apache, etc. or a set of keywords derived from the original post title and text. The absence of any of these words is a clue that the post just might be SPAM or otherwise off topic and should be placed on moderation for the admin to review.
Since I've started using this new 'lack of content filtering' technique it's snared the few hand submissions to my other site that were completely off topic, those that I would've deleted immediately. The beauty is I can continue to leave the posting wide open for humans, not moderate everything, with only those posts that don't match the topic getting instantly set to moderate.
I expect a few false positives but so far 'lack of content filtering' is doing exactly what I expected it do and set a couple of crap submissions last night for shit like "zanaflex information", apparently some pill I've never heard of and "News, Stores, People, Careers at Finditt", some wannabe search engine, to moderate automatically while letting 20 on topic things thru without a hitch.
Another automated weapon in the war on spam!
GoodBidWords.com Scrapes LookSmart
Noticed at hit from one of my scraper probes in GoodBidWords.com which contained the IP address of the original crawler.
Looked up the IP address and guess where it came from:
"Mozilla/4.0 compatible ZyBorg/1.0 (wn-14.zyborg@looksmart.net; http://www.WISEnutbot.com)"Isn't this precious that GoodBidWords got caught because of all the places to scrape they decided to scrape a search engine that I don't permit to crawl my site!
What a hoot, second-hand scraper busting, this rocks!
Tuesday, October 03, 2006
phpBB Membership Spamming for Authority
We first reported about phpBB spamming the other day when we stumbled upon this "DISY registration spamming script" and since then have had a little time to examine what spammers are doing with phpBB trying to gain authority.
Let's just check a few of these spammers in Google:
pimpdomain.netHell, just try any of the domains listed in my Technorati Loves Spam post and search for the domain name and phpBB and see what shows up.
thewestgategazette.com
ritalin-pharmacy.com
Just amazing what these assholes do with this shit cluttering up the net with spam.
Technorati Loves Tasty Cloaked Blog Spam
I've noticed that Technorati has been happily eating up scraped and cloaked blog spam for ringtone sites, among other things, like it's fucking candy.
Let's use a search on my blog name as an example:
Click on those links and it's always to the same spammy page name like these hosted on theplanet.com of course:
http://artinexis.net/#comment-341One server is 70.87.88.121 or 79.58.5746.static.theplanet.com with these domains all spewing ringtone ads:
http://themetrogiant.com/#comment-329
http://pimpdomain.net/#comment-341
about-levitra.netAnother annoying server is 70.87.88.108 better known as 6c.58.5746.static.theplanet.com which has these goddamn domains:
acvfa.net
artinexis.net
cariculture.net
catsfive.net
citadel1.net
cloudsite.net
eightonefive.net
rennenmotorsports.net
t3linkcom.net
talonpro.comOr this fucking spam filled server host 70.87.88.106 hosted by our fucking friends 6a.58.5746.static.theplanet.com:
tempuspercussion.com
terminal34.com
the-god-poll.com
theincrediblesuckingspongies.com
themetrogiant.com
thepulse2000.com
thespinet.org
thewestgategazette.com
thoweu.com
tlc-express.com
perseidslive.com
pimpdomain.net
poemnet.net
posses1consent.com
projhind.com
ptcsucks.com
r1g4t2you.com
rbigkitty.com
rep1icas.com
ricohtour.com
rising7.com
ritalin-pharmacy.com
Here's the same shit about ringtones they all show:
How about you just set the phone on buzz, stick it in your pocket, and you'll never miss a call or be confused it's someone else's phone ringing, and best of all you can do it without lining the pockets of the cell companies or perpetuating this spam. Better yet, just shove that phone up your ass as most people that feel the need to never miss a call by using goddamn custom ringtones are probably talking out of their ass anyway. While you're at it, shove some custom phone face plates and a nice blue tooth headset up your ass too, but I digress.
WAIT A FUCKING MINUTE...
I think I see a pattern here 70.87.88.106, 70.87.88.108, 70.87.88.121...
Let's try 70.87.88.120 and see what we find:
asmort.netMore ringtone spam spam spam....
bevirusproof.net
conlajusticiaysociedad.net
fabionne.net
friendshipmotorinn.net
macoszone.net
palick.net
phila-ibiz.net
themikecam.net
wesmn.net
Or let's try 70.87.88.115:
audio-wire.net
buy-cheap-2u.com
chabadofbuffalo.com
cheap-online-buy-free.com
el-condor-pasa.net
ellemtel.net
fairy-wings.net
free-top-sex.net
gotobiz.net
healthcybeline.com
javabooks.net
jemison-nealon.net
lolitasexlinks.net
macromediaseminars.com
netnetn.net
remax-powell-m-corpus-christi.com
stopsundiata.com
wangmatongli.com
xinyifang.net
Yes!
More spam spam spam spam spam!
OK, this is obviously a big operation with lot's of shit domains serving up spam on lots of IP's, I'm bored with this already, if you want to help fill in more blanks with this ringtone spammer go to Domain Tools Reverse-IP page and type in the IP address in that range and see what's on the servers.
Maybe they should change their name to Spamorati as they seem to love these fake blogs reposting old posts.
BTW, if you need help with automating the identification of spam over at Technorati just drop me a line as I'd be more than happy to show you how to automate the process for a small fee!
The things I could teach them on ways to clean up their listings and improve their service would boggle their minds.
Saturday, September 30, 2006
ShoeMoney's Blog Spam Stopping Primer
The day after my battle cry to Rally the Anti-Spammers here comes ShoeMoney with some great suggestions for stopping blog spam. Everything ShoeMoney posted is very solid advice but some spammers have already been evolving past some of those patches which is why I use my draconian anti-spam methods. Basically, ShoeMoney's advice will stop the majority of your garden variety spammers, but not all as they are constantly adapting, so as you improve your defenses they improve their ability to bypass those defenses.
Remember, security is built in layers and the more layers you pile on, the more the spammers will chip away at your security so building the better spamtrap just results in smarter spammers and they're already here which I'll address with examples below.
Let's examine ShoeMoney's anti-spam advice, see what some state of the art spammers are already doing, and add a few more tricks here and there for even better security.
Starting with the first item he listed:
5) Deny Access to No Referrer Requests
The approach does work on most spammers but I had about 10 requests today where it would've failed. Not that you shouldn't implement this, it's a good trick to stop a lot of spam, just be aware it won't stop everything.
Example:
My bounced spam log shows the following:
IP: 84.110.248.226Take a look at what's in my server log:
User Agent: "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
Subject: "Viagra"
URL: http://anol.webhosting.gs/viagrageneric.html#viagra
84.110.248.226 - "POST /formsubmit.html HTTP/1.0" 200 11918 "http://www.mysite.com/formsubmit.html" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"Yup, that's right, a referrer, and I had about 10 of those and they were all from spambots.
Stopping the poorly coded spambots is easy, but they won't be vulnerable for long as the patch to add the domain name being spammed into the referrer is trivial so I expect this anti-spam advantage to be short-lived but I use it too, you should still do this.
Now, let's tackle the next item, which is VERY good advice:
4) Kill tor anonymous proxies
I block many proxies on my servers, which does stop a lot of spam, but don't think that all spammers use known proxies. This is the reason I also block dedicated server hosting facilities because a series of $2 webhosting accounts can be used to effectively spam and bypass the proxy lists.
Example of 4 sample spams (out of many) today that all had referrers mentioned above and came from some ISP/Host called bezeqint.net:
09/29/2006 84.110.248.226Use this with caution:
"Viagra" http://anol.webhosting.gs/viagrageneric.html#viagra
09/29/2006 84.110.244.240
"Viagra" http://gerda.forospace.com/#viagra
09/29/2006 84.110.243.107
"Cialis" http://borea.forospace.com/#cialis
09/29/2006 84.110.241.163
"Cialis" http://kaizer.webhosting.gs/cialisbuy.html#cialis
2) Blacklist Repeat Offenders:
First off, blacklist on the FIRST offense so there is no second time. However, you really need to know what you're doing and lookup who the IP address belongs to so you aren't blocking IP addresses from places like the AOL IP pool (reused every 15 minutes or so) or any other shared proxy dial-up IP pools as those IP assignments are very temporary and the next access is probably a different visitor, not a spammer, so be very careful with this.
This is a gem and we can make it better:
1) Rename your comment file
Excellent advice as I've done that on some websites but don't be shocked when it's short-lived as spammers also have crawlers looking for these comment pages and the fact that you're still linking it under the keyword "comments" is a dead giveaway.
If you're going to change the file name, also change the word that links to the file name to "discussion", "verbal intercourse", or "rants", anything but "comments" to throw them off.
Additionally, move the actual FORM into obfuscated javascript document writes. How this works is the spambot scanning your website can't even find the webform to submit comments as most bots don't use javascript, so only an actual visitor would see an actual webform written into the web page via javascript.
Don't forget the CAPTCHA!
Now, the one thing ShoeMoney didn't mention which works wonders is a simple CAPTCHA and it's keeping a few of my sites spam free without ANY other work involved. Yes, there are ways to bypass a captcha but it's not easy for the spammer. So far most captcha protected sites are safe with such simple protection, but I expect that situation to escalate soon.
Kudos to ShoeMoney for spreading the word, we need more anti-spam information spreading and more people jumping on the anti-spam bandwagon so we can rid the 'net of this scourge as soon as possible and move on to more productive activity.
Thursday, September 28, 2006
Virginia Tech's Computer Science: Wiki Spam 101
My website stops spam posts cold, and logs them, so that eventually I can glance over the list of bounced spams now and then just to see what was caught and this one was priceless:
I looked and thought, "Viagra spam linking to VT.EDU? Could their server be hacked like SpamHuntress is posting about?" So I click the link and of course it uses VT.EDU's server to redirect me to some viagra sales site just like the URL would make you think it would, no surpise there.09/28/2006
200.88.223.98
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
Subject: "Viagra"
URL: http://research.cs.vt.edu/advance/tiki/
tiki-directory_redirect.php?siteId=3284#viagra
So I trimmed the URL to see what in the heck this site was and it's ADVANCE, FOR THE ADANCEMENT OF WOMEN IN ACADEMIC SCIENCE AND ENGINEERING CAREERS and it's full of advances for MEN such as viagra, cialis and levitra spam plus a whole bunch more.
Well, the IT dept. and professors in charge of the VT computer science program should probably start quaking in their boots as I would be VERY UNHAPPY if I was the Dean.
This is completely unacceptable when the IT guys and CS Profs aren't using even rudimentary anti-spam technology like, oh, maybe a simple CAPTCHA to stop this shit.
I want my tuition refunded.
BTW, whoever these spammers are, they've been VERY BUSY little beavers.
Time to Rally the Anti-Spammers
After the demise of Blue Security and this recent meaningless default judgement against SpamHaus, the spammers are getting braver and bolder by the day. Now, one of the most vocal anti-spammers around, SpamHuntress, has recently come under attack after exposing a few people that really didn't want to be exposed.
Even one self-professed blackhat SEO web spammer has the audacity to tell SpamHuntress to "get a life" because she must be cutting into his livelihood. Maybe I'm just too lazy, but who would've ever thought of registering for a bunch of forums and never posting as an SEO tactic? Using his DISY registration spamming script probably sped it up and he's busy making friends [scroll to bottom] as well.
OK, so now the phpBB people will need to be alerted to add NOFOLLOW to all those links in the registration page to stop this SEO vulnerability, but I digress, will rant about that later.
Unlike email spam, which is a real pain in the ass to stop, there is absolutely no reason we have blog, forum or guestbook spam whatsoever except for shitty programmers writing the stuff and people using it that either:
- have abandoned their websites or forgotten that old guestbook or blog now littered with junk
- aren't aware there is a problem as many spambots post on older threads
- don't know there are solutions to these problems
- aren't capable of installing the patches even if they are aware of the solutions
However, for those that like to continue to do things the hard way, here's a list of software you can install to stop the spammers:
- Askimet which is a networked anti-spam product which is overkill when less than 20 lines of code can bounce current web spam, but some people seem to think it's OK so I'll mention it for those that like this approach.
- Bad Behavior which is a script that analyzes spambot behavior on a blog and shuts them down, but may get a few false positives.
- Math Comment Spam Protection which is a simple captcha that requires simple math to post which is very popular these days, but I'm afraid may be the easiest to defeat ultimately.
- Check out this list of anti-spam solutions for WordPress.
Please point them to a resource to solve the problem or offer to help them add the plug-ins pro-bono or for a nominal fee if they don't understand how, or if all else fails alert the host to help sites overflowing with spam and see if they'll be of any assistance.
Don't forget, the purpose of these spammers is to drive direct traffic and also get results in Google so when you stumble upon these sites in Google, make sure you file a Google Spam Report while you're there to get them whacked from the search results.
We can stop this in the next year or two, as long as people quit being complacent and just install the upgrades, patches, captchas and other anti-spam tools.
Spread the word, let's just get this done so we can stop talking about it already!
Wednesday, September 27, 2006
MySpace: Porn Networking Spam Machine?
The other day I signed up for MySpace while researching the members with "Click The Ads" on their pages encouraging others to commit click fraud to fund their various lame causes.
Unfortunately, signing up for MySpace immediately resulted in a couple of porn spams sent to my Inbox which really pissed me off.
So I get some shit that looks like this:
FROM: MySpace Events
SUBJECT: .. has invited you to: I seen you online
Hi ,
.. has invited you to an event on MySpace:
Click the link below to view the event details:
http://events.myspace.com/index.cfm?fuseaction=PORNSPAM
Now below this, there is some bullshit message from MySpace:
At MySpace we care about your privacy. We have sent you this notification to facilitate your use as a member of the MySpace.com service. If you don't want to receive emails like this to your external email account in the future, change your Account Settings to "Do not send me notification emails."Really, you care so much about my privacy you let goddamn porn spammers send me fucking email?
I'm touched, a tear comes to my eye ...
... yes a tear, because I realize can't reach out and smack whoever let this shit happen upside the head!
Anyway, here's the website on MySpace linked from the spam:
Here's the first site's link to a girl with a webcam:
And here's the second spam site's girl with a webcam:
I'm wondering if people under 18 get these spams too?
I think I'll just cancel the account because MySpace is no place I was to be associated with.
Monday, September 25, 2006
MySpace: A Click Fraud Social Network?
Maybe it's just Web 2.0, or Web Welfare 2.0, but it appears that stealing from advertisers is now something that is accepted in social networks. Let's look at what we find on sites like MySpace and others which are a good place to build up a nice list of friends to click your ads, especially the Google ads, because we all know that friends click friends ads, especially if you want your friends banned from AdSense.
Even on YouTube where people can't put up their own ads they beg people to come to their website and click the ads to support them putting up more videos!
The most shocking is Blogger, which is owned by Google, the creator of AdWords and AdSense, which hosts sites that encourage people to "Click the Ads" to defraud the very advertisers they rely on for their massive income.
How difficult would it be to have a single employee out of the entire Googleplex devoted just to keeping click fraud off their own property?
You know the answer, I know the answer, yet a simple search reveals that it's not being done, or not done adequately at any rate or there would be no sites returning results from Blogger on this topic if they were on top of the problem.
The technology for these sites to deploy an automated process to locate pages within their sites that contain calls to "click the ads" or "click Google ads" or any combination and eliminate this fraud on a daily basis is so trivial and rudimentary that beginning programmers could do it.
Bottom line is there's absolutely no excuse for this type of call for advertiser click fraud to be allowed unchecked on these sites, not in MySpace, YouTube, Blogger, Google, Yahoo, MSN or anywhere else and why Click Fraud 2.0 continues to perpetuate on the web when it's so easy to thwart frankly boggles the mind.
Flickr Member Requests Click Frauding Advertisers for the Children
Well, I've seen all sorts of excuses to advocate click fraud but the plea on Flickr to commit a crime for the children is a new one and more despicable than any I've seen before. Think about the precedent that this sets in impressionable young minds that it's "OK TO STEAL FOR A CAUSE" when crime is never OK. Sadly, all of the good this person has possibly done for these children was wiped away with one call to arms to defraud people for a cause.
If you want to save the children, set up a Paypal account and teach the children than they can be helped by the generosity of others, not by others commiting FRAUD!
Here's the screen shot from Flicker:
And the site it lands on in Blogger:
Come on buddy, just ask for donations and keep it legal as we all love the children but this is over the top.
Saturday, September 23, 2006
Search Engine Spammers Extraordinaire
OK, these idiots made the classic mistake of scraping one of MY pages so they're about to get outted in a massive way. Unfortunately, in this case I didn't get an IP address and my content was already missing from their site thanks to the slow crawl and index of MSN, but a little research proved this was a HUGE operation of mind blowing proportions.
I got bored checking all the domains as some are hosted in the same place, some aren't, too many to look at but it's all spam. Perhaps the same person, or perhaps a bunch of idiots running some automatic website generating tools.
The sites tend to come in 3 flavors, AdSense monetized articles, AdSense monetized scrapers sites (scroll WAY down) and AdSense + Shareasale sites.
Just search for the phrase "When we had a difficult think about this project" in Google, Yahoo or MSN and you'll see a shitload of pages from these search engine spammers.
Also, try a search for the phrase "Foraging for the best file on" in Google, Yahoo and MSN and see more shitloads of pages.
You can see all sorts of key phrases these sites repeat and bust more and more of them like this "Everyones path is incomparable and everyone" one on Google or Yahoo.
And even more shit like "If you've worked with a portal" on Yahoo.
Someone noticed their terms were hijacked in these bullshit pages and blogged about their suspicion on what's going on.
Seriously though, I bet I could write a script to identify and locate all the bullshit spammers using this data with all their common phrases as it's so easy to spot once you have a data sample like these to analyze.
Spam, spam, fucking spam, and not so smart fucking spammers.
Whitelist OPT-IN htaccess file
People are always asking me how to build an OPT-IN .htaccess file, which I advocate, opposed to the traditional blacklist methods.
The problem with OPT-IN is it's VERY unforgiving and you really need to check your visitor stats and make sure you're letting in all the crawlers that are sending you traffic.
Belows is a bare bones sample of how it works and anything not in the list gets a 403 Forbidden error so you'll probably need to add more items and refine this for your particular website.
Sample .htaccess file for Apache 2.0:
#allow just search engines we like, we're OPT-IN onlyJust save the above as a file named ".htaccess" in your httpdocs or root web folder in your hosting account and all the crazy bots abusing your site will get bounced from now on.
#a catch-all for Google
BrowserMatchNoCase Googlebot good_pass
BrowserMatchNoCase Mediapartners-Google good_pass
#a couple for Yahoo
BrowserMatchNoCase Slurp good_pass
BrowserMatchNoCase Yahoo-MMCrawler good_pass
#looks like all MSN starts with MSN or Sand
BrowserMatchNoCase ^msnbot good_pass
BrowserMatchNoCase SandCrawler good_pass
#don't forget ASK/Teoma
BrowserMatchNoCase Teoma good_pass
BrowserMatchNoCase Jeeves good_pass
#allow Firefox, MSIE, Opera etc., will punt Lynx, cell phones and PDAs, don't care
BrowserMatchNoCase ^Mozilla good_pass
BrowserMatchNoCase ^Opera good_pass
#Let just the good guys in, punt everyone else to the curb
#which includes blank user agents as well<Limit GET POST PUT HEAD>
order deny,allow
deny from all
allow from env=good_pass
</Limit>
Remember, anything not listed will no longer have access so be careful and make sure everything your site needs allowed is in the list.
Enjoy.
Googlebot Validation
Google has finally completed a DNS project that will allow us to use a simple reverse and forward DNS check to verify it's really, truly, honestly Googlebot and not a cheap immitation, or Google crawling thru a proxy, or anything else you can imagine.
I'm so sick of explaining why you might need this and what it solves you'll just have to follow a few links and read the threads at these various places.
Here's the official How To Verify Googlebot post on Google's blog.
Then you can check out what's been said about How To Verify Googlebot on Matt's blog.
Then a couple of threads on WMW about Verifying Googlebot that should answer any other questions on this topic.
Thanks again to Matt for getting this project finished!
Tuesday, September 19, 2006
How Important Are Plurals
Many people ignore plurals when they optimize their website and miss a lot of opportunity for additional search engine traffic.
Here's a few trend examples:
Take a look at plumbing, plumber and plumbers and you'll note that the plural is just as often the search term as the singular plumber.
How about teaching, teacher and teachers where all 3 run very close and teachers appears to dominate the search trend by a thin margin.
Last but not least, something closer to home with blogger and bloggers, where blogger clearly stands out as the dominate term but bloggers is statistically significant enough to merit ranking for the plural.
So don't forget to rank for your keyword plurals or someone else will rank there instead of you and they will KICK YOUR S!
Request from India
Just when I thought it was going to be a boring day I got a link-exchange spam from one of those wonderful Indian SEO's that wouldn't know how to promote a website to save his own life.
I'm actually shocked this email didn't include the usual threat that "you have 24 hours to confirm a reciprocal link before we remove yours from our site".
Boy, doesn't this shit look familiar:
Dear Webmaster
Greetings from India
Happened to visit your Webpages : [FILL IN BLANK OF SPAM RECIPIENT HERE] & liked it very much.
Would like to request you to have a look on our site :: [FILL IN BLANK OF SITE BEING SPAMMED HERE]
Hope you'll like this site. We are trying our best to spam the shit out of everyone in the name of India, You can help us by just adding our link on your wonderful website. And these exchanging link with good quality websites is beneficial for both the site to get a good ranking in search engines & that will help both of us in driving Traffic.
So We request you to add our link at your Website
Here is the Link Information of our Sites ::
URL : [LINK TO OFF TOPIC SHIT GOES HERE]
Link Text : We Spamma U Ass
Desc. : That's Right, This is Spam, its no more a dream!!
Just do let us know if this acceptable for you.
Hope to have quick & positive response.
Thanks in Advance
Best Regards
Sendjay Sumspam
Spamming-Our-Ass-Off.com
BTW, if you're the Indian fuckhead sending this shit, FUCK NO I WON'T LINK TO YOUR SITE you goddamn moron.
Just a lovely way to start the day.
Say it with spam.
Thursday, September 07, 2006
Counting Scrapers on your Abacus
Had a couple of persistent little fuckers hosting with Abacus that just keep trying and trying to download a boatload of pages that I've been monitoring for months now.
The specific IPs of these boxes are:
206.225.82.155 "Mozilla/4.0 (compatible ; MSIE 6.0; Windows NT 5.1)"Toss in a couple of proxies:
206.225.91.164 "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)"
206.225.83.179 "Evaal/0.7.2 (Evaal search engine; http://evaal.coml; bot@evaal.com)"
216.55.161.38 "Java/1.4.1_04"
216.55.142.118 "Mozilla/4.0 (compatible ; MSIE 6.0; Windows NT 5.1)"
216.55.162.3 "PEAR HTTP_Request class ( http://pear.php.net/ )"
216.55.147.80 "sna-0.0.1 mikeelliott@hotmail.com"
206.225.85.127And some other miscellaneous bullshit not worth mentioning.
206.225.86.86
Here's what to block:
OrgName: Abacus America Inc.Now you've been COMPLETELY BLOCKED so count THAT on your Abacus!
OrgID: ABAC
NetRange: 206.225.80.0 - 206.225.95.255
OrgName: Abacus America Inc.
OrgID: ABAC
NetRange: 216.55.128.0 - 216.55.191.255
More Evolving Scrapers
Like I've been reporting, they're all going stealth.
I keep seeing user agent change from this:
62.163.33.234 "Java/1.4.1_04"To this:
62.163.33.234 "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)"Soon the usual blocking methods won't work whatsoever.
Wake up and smell the COPY before it's too late!
Block the Bots Tonight
Time for a little lunacy break for people feeling blue battling the bad bots.
Sing along boys and girls...
Sung to the tune of "Rock Around the Clock"
with apologies to Bill Haley and the Comets.
One, two, three bots, four bots, blocked.
Five, six, seven bots, eight bots, blocked,
Nine, ten, eleven bots, twelve bots, blocked,
We're gonna block all the bots tonight.
Put your firewall on and lock em out,
We'll have some fun when they scream and shout,
We're gonna block all the bots tonight,
We're gonna block, block, block, their scraping blight.
We're gonna block, gonna block, all the bots tonight.When the block strikes two, three and four,
If the scrapers slow down we'll yell for more,
We're gonna block all the bots tonight,
We're gonna block, block, block, their scraping blight.
We're gonna block, gonna block, all the bots tonight.
When the server dings five, six and seven,
We'll be right in bot blocker heaven.
We're gonna block all the bots tonight,
We're gonna block, block, block, their scraping blight.
We're gonna block, gonna block, all the bots tonight.
When it's eight, nine, ten, eleven too,
When the counts hit twelve, we'll laugh and yell,
I'll be blocking bots and so will you.
We're gonna block all the bots tonight,
We're gonna block, block, block, their scraping blight.
We're gonna block, gonna block, all the bots tonight.
As a dozen bad bots have just went to hell!
We're gonna block all the bots tonight,
We're gonna block, block, block, their scraping blight.
We're gonna block, gonna block, all the bots tonight.
