Monday, January 16, 2006

Blank User Agents

Who in the hell is using blank user agents on the web?

I can't tell if these are SpamBots, Scrapers, Trackback Pings, all of the above but none of them are getting thru so it doesn't matter.

They must naively think if people block per user agent string that if you have no user agent string you'll slip thru the cracks which might've been true until my paradigm shift switched from blacklisting to whitelisting.

You people still blacklisting, and you know who you are, are wasting your time on a no-win scenario that just doesn't work.

2 comments:

Anonymous said...

I seem to remember that Norton's blocks the user agent so it shows up blank, and this is set by default.

I had some problems with this on one of my sites where I was using hotlink protection for my images, and people using Norton's couldn't see the images because no user agent string looked like a hotlink request.

IncrediBILL said...

That's an interesting theory to investigate - considering I earn my money via advertising and Norton Firewall blocks ads from showing this could be a blessing - 2 birds, 1 stone.

I'm pretty sure I'm seeing real scrapers/downloaders though as a human with Norton Firewall wouldn't blindly try to download 500 pages in 2 minutes.