Most of you probably think I'm very brash and run amok implementing things on my server all willy-nilly with hardly a concern for the damage that I might be inflicting on my visitors but that's further from the truth than you can imagine. I'm actually very cautious and do a lot of testing with each new approach I phase into my bot buster by first executing the rules and giving me a preview of what would happen for a day or so before I make the rules live.
That means to date all the IPs I've been banning are being banned in software so that I could monitor their returns and activity to verify they're really a permanent source of abuse or a one-shot attack from a dynamic IP.
Well, enough of them are returning on a regular basis that I've decided it's time to start the next phase of the project which I call Maxwell's Silver Hammer where it will decide automatically that the source of abuse is bad enough and just drop them in the .htaccess file so they simply bounce off the server and don't even tie up my scripts keeping an eye on them anymore.
So here we go sweating bullets that this code won't accidentally crash some night and leave the .htaccess file all banged up and bring the site down to it's knees.
Progress, gotta love it.
Wednesday, March 08, 2006
Maxwell's Silver Hammer
Subscribe to:
Post Comments (Atom)
8 comments:
Ick your banning ip's with htaccess ?
I had a terrible time with that killing my server response time when I banned all of China. That was quite some time ago
Well I don't plan on putting a ton of IP's there, just the most abusive as I'll probably save a few GB in bandwidth alone letting them bounce off. The plan is to try blocking them for a period of time and then remove them again to see if they took a hint or not.
Problem is that the linear lists used in firewalls and Apache to allow/deny never took into account the sheer volume of assholes we have today and a simple pre-processing of the .htaccess file to make a binary counterpart to be used on subsequent accesses with the list of IP indexed for speed would easily resolve this.
Not my fault the people that build these tools can't code worth a shit or think outside the box.
Hell, I applied a hash index table on top of DOS 3.x directory (entire drive) for a project once and you could locate any file on the hard disk in a fraction of a second way back in the early 80s on an pokey assed 80286 and these ass monkeys writing Windows and Linux still don't have directory searching optimized.
I gave up trying to fix things years ago so I deal with what I can change and to hell with the rest.
Yeah I had a few thousand lines hahahaa apache no likey
Neways I ended up writing a function I could call in my scripts to test if it was good or bad. Stored it all in a mysql table. Worked better for me at the time and was more scalable.
I concur with you as I'm using a database to track them now too and it's much more efficient.
The problem is the worst offenders can still engage a lot of CPU horsepower beating on the site for many hours at a time so JUST the most vile offenders will start bouncing off apache.
Have you tried reconfiguring your firewall on the fly ? Would blocking at your firewall be more efficient ?
When I was doing it, I wasn't really brave enough to try it because I had a single server and only one IP ( I was a cheap bastard lol ).
Also I was just fighting proxy users from posting on my forum not scrapers, so the situation was a lot different.
You know the dates are all wonky on the comments.. weird.. mine are showing this date on the comments "8/3/06 5:36 PM"
Remember that the purpose of the design of my script is to be able to drop into a standard web hosting account, one where access to server-wide firewall rules aren't available so .htaccess is the only hard block available to your average webmaster.
BTW, I de-wonked the dates for you but now some UK people will bitch about it being all americanized and shit.
Oh ok I thought this was just for you.
Post a Comment