I’ve just spent the better part of a week troubleshooting server performance problems for one of my clients. They’re running a number of sites on a dedicated server, with plenty of RAM and CPU power. But lately the sites have been really slow, and the server has frequently run out of memory and started the dreaded process of thrashing.
Fearing inefficient code in cms34 may be to blame, I spent a few days trying to optimize every last bit of code that I could, which did make a slight improvement, but didn’t solve the problem.
Then I spent a few more days poring over the Apache configuration, trying to optimize the prefork settings and turning off unnecessary modules. Still, to no avail, although getting those prefork settings optimized, and thus getting Apache under control, did allow me to notice that MySQL was consuming CPU like mad, which I had previously overlooked.
Hmmm… that got me thinking. I fired up phpMyAdmin and took a look at the running processes. Much to my surprise, almost every MySQL process was devoted to an abandoned phpBB forum. Within moments I realized the forum must be the source of the trouble, which was confirmed when I found that it had over 500,000 registered users and several million posts, almost all of which were spam.
As quickly as I discovered the problem, I was back in the Apache configuration, shutting down the forum. Then a quick restart of MySQL (and Apache, for good measure), and the sites were faster than I’ve seen them in months.
The moral of the story: if you have a web server that suddenly seems to be grinding to a halt, don’t spend days optimizing your code before first looking for an abandoned forum that’s been overrun by spammers.