Tuesday, January 24, 2006

Not So Fav Icon

OK, when I pull a major blunder I do it right as minor fuck-ups are for amateurs.

Follow along carefully as this will all make complete senselessness eventually...

This problem all started sometime recently as it appears somewhere along the line my favicon.ico got zapped off my server and I never noticed nor did I bother looking at the logs close enough or I would've figured this out a few months ago.

Now imagine that my Apache configuration doesn't know the favicon.ico is an image and on a 404 error was actually displaying a 404 page for the missing icon.

Next, trying to capture more visitors instead of letting them see a 404 page and leave, possibly because of site maintenance errors, the 404 page at some point was redirected to my home page.

Last but not least, I changed my mind a few days ago and put my bot stopper code on the home page after seeing what scrapers could do with just that little amount of content.

Suddenly a small rash of people got banned with about 30 page views in 5 seconds. Looked at what was happening and these people hadn't downloaded 30 page views but had a shitload of requests to favicon.ico which looked very odd. Must be getting dense as it took a couple of days of seeing this before my little brain said "note the favicon.ico requests getting a 404 error".

Let's see what's going on here and try it:

Up pops the home page!

Oh fuck.

So a browser or something asking for the favicon.ico about 20 times in a row loaded 20 404 pages which redirected to 20 home pages which tripped the scraper alarm and stopped them from accessing more pages temporarily.


Uploads favicon.ico, tucks tail between legs, hides quietly in the closet until the massive wave of embarassment passes.

No comments: