UNDERDOG of PERFECTION

I’ve been having some issues with a particular server lately where it keeps going down. I probably should have given it serious attention sooner, but it’s a “personal” server (runs this site, my wife’s blogs, and a few other sites I’m hosting as a courtesy to friends), and I’ve had a lot going on lately.

This morning it seemed to be worse off. MySQL just wouldn’t start. My monitoring script that fires off every 10 minutes so I don’t have to be on-call 24/7 was doing its best, but it just kept restarting in vain.

Time to look into the issue. I found that MySQL was running, consuming over 100% CPU (it’s a multi-CPU machine so the maximum percentage is over 100), but nothing was loading.

Running systemctl status mysql.service showed this, which kind of surprised me:

Status: "Server startup in progress"

So, something was causing MySQL to just get stuck in the startup process and never actually get up and running. I figure that is usually a corrupt database, which could be a nightmare, especially since I’ve been ignoring this issue for a week or so. Having to restore from a week-or-more-old backup would be a minor inconvenience to me, but my wife writes a lot on her blog and she would not be happy to lose several days of work.

I needed to check out the MySQL log file. At 17 GB — yikes — that meant using tail to just check out the last few hundred lines.

Here’s something interesting:

2025-06-08T14:33:14.651332Z 1
[ERROR] [MY-011899] [InnoDB] [FATAL] Unable to read page [page id:
space=0, page number=5] into the buffer pool after 100 attempts. The
most probable cause of this error may be that the table has been
corrupted. Or, the table was compressed with with an algorithm that is
not supported by this instance. If it is not a decompress failure, you
can try to fix this problem by using innodb_force_recovery. Please see
http://dev.mysql.com/doc/refman/8.0/en/ for more details. Aborting…
2025-06-08T14:33:14.651347Z 1 [ERROR] [MY-013183] [InnoDB] Assertion
failure: buf0buf.cc:4110:ib::fatal triggered thread 140583111841344
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

I went straight to the last link to the MySQL docs, Forcing InnoDB Recovery. Since this site has a bunch of WordPress databases that all use InnoDB tables, that would — hopefully — be the solution.

It’s pretty simple, once you find the right configuration file to put this line of code in:

innodb_force_recovery = 1

My server is running Ubuntu, so the file I wanted (I had to hunt around a bit) was:

/etc/mysql/mysql.conf.d/mysqld.cnf

I added that line at the end of the file, ran systemctl start mysql and much to my surprise, after about 3 seconds, the command prompt returned, with no errors. I fired up Safari and checked out my site and… well, since I’m writing this here, you can guess the rest.

Of course, is this really a solution? I was hoping so. The name of the parameter sounds like it’s, y’know, going to fix any problems it encounters. But reading the documentation further, it looks like it is really designed just to bypass certain safety mechanisms in order to allow the system to run so you can do your own troubleshooting.

Unfortunately I’m not quite sure where to begin with this troubleshooting. There are over 30 databases on this server, so I’m looking at somewhere over 500 tables, any of which could be the culprit, and the log files don’t give any indication of which table — or even which database — is the source of the problem.

So, when in doubt, I like to start as simple as possible. Since innodb_force_recovery is supposed to be only a temporary setting and it limits certain functionality, I knew I would eventually have to turn it off again. Let’s just try that now and see what happens.

I commented out the line I had just added to the config file, tried restarting MySQL, and… it worked. I’m not sure if starting up with innodb_force_recovery did do something that cleaned up the problem, or if just using that setting to get past whatever was hanging things up before allowed the normal boot process to do some standard cleanup, but in any case, it seems to be working fine now.

But if I get another alert that things have gone down, I’m not going to wait a week to investigate this time, no matter how much more pressing work I have going on.

For years, variations on this have been my go-to CSS “clearfix” approach:

.clearfix::after {
  clear: both;
  content: '';
  display: table;
}

It’s the “conventional wisdom” on how to do this. But I found recently I was trying to do it and… it wasn’t working. In the middle of trying to figure out why it wasn’t working, it struck me that with modern CSS there’s probably a better way to do it. Why not this?

.clearfix + * {
  clear: both;
}

It doesn’t require the trickery of creating an empty content block on a pseudo-element. It’s less code. It seems like exactly the kind of situation the CSS + selector is intended for.

One little snag — although as I recall this was a snag with the old method too — is that it doesn’t introduce any spacing between the elements, and even putting a margin-top on the latter element doesn’t have an effect, although padding does.

So, it’s not ideal… or maybe there’s just something else I’m overlooking. But when is anything in CSS ideal? And how often isn’t there something I’m overlooking?

I’m mainly writing this here so I’ll find it again in a few years when I need to do this and can’t remember how I did it before.

Say you want a nice looking, centered text header integrated with a horizontal rule, with the line on both sides of the text and a bit of a gap, like this:

Here’s My Nice Header

The only HTML involved there is this:

<h4 class="my-nice-header">Here's My Nice Header</h4>

Besides Flexbox, obviously, I’m leaning heavily into CSS Pseudo-elements to make this work. It occurred to me immediately that I could use the ::before and ::after pseudo-elements for the lines, but my real flash of insight (at least it felt like a flash of insight to me) was using the ::first-line pseudo-element to apply Flexbox styling to the text itself, without needing anything like a nested <span> tag.

Here’s the exact CSS code I’ve included in the page to render this header:

.my-nice-header {
  align-items: center;
  display: flex;
  gap: 1rem;
  width: 100%;
}
.my-nice-header::before, .my-nice-header::after {
  background: gainsboro;
  content: '';
  display: block;
  flex: 1;
  height: 1px;
}
.my-nice-header::first-line {
  display: block;
  flex: 1;
  text-align: center;
}

Note: I removed some font styling and margins that aren’t pertinent to the actual “trick” itself. I left in the background color, because you need to specify some color for the lines not to be invisible.

I watch a lot of YouTube. Most of what I watch on YouTube is related to music or video games, but I also have a penchant for videos about cooking, architecture, and the history of 20th century technology. A couple of my favorite channels are Tasting History with Max Miller and Phil Edwards.

By this point all of the algorithms know I am a 50-something GenXer, with a moderate affliction of nostalgia. So as much as I know listicles (or the video equivalent) are clickbait… well, I take the bait.

So, you know just as well as the algorithm did that I would not scroll past a video called “15 FORGOTTEN Sandwiches That FADED From Your Family Table.”

Go ahead, watch it.

I was struck immediately by a few things: first, the weird overuse of “aged film” effects on the apparently stock video clips that vaguely corresponded to the sandwiches being described.

Next, I noticed a weird monotony to the narrator’s delivery. I didn’t like it, but I figured it was just his style.

But that was when things got weird. Out of nowhere, the introduction of the “Mock Ham Salad Sandwich” was spoken in a different voice, with a strong Asian accent. Then it was back to Mr. Monotone.

Suddenly it all clicked. This entire video was AI generated.

I’m not sure if the video content itself is AI-generated, or if it’s just… um… AI-concatenated. I didn’t scrutinize it super closely, but I didn’t see any of the telltale signs, like mangled text, deformed human hands, objects spontaneously transforming into something else.

It might all just be stock footage. But a) I do know AI-generated video has improved a lot recently, and b) it also seems unlikely that they’d have found enough marginally-relevant (and some of this is very marginal) stock video footage for each of these sandwiches.

I was struck specifically by the pimento cheese sandwich, and how the shot of it shows a paper wrapper with the Masters logo. Yes, the pimento cheese sandwich is inextricably tied to the Masters golf event at Augusta National in Georgia. Why wasn’t that detail mentioned in the narration? (It’s probably worth noting that this is a fact I’m able to recall only because I saw a Facebook post about it a few days ago.)

The channel that posted this video only has two videos, both posted this week. Combined, they have fewer than 1000 views.

Both of their videos share a similar format. But what really freaked me out was that my YouTube home page also had another video from a different channel that was similarly nostalgia-stoking and, based on the little preview that played, had the same telltale “aged film” effect.

Who is behind this crap? How much worse is this all going to get?

I’ve been meaning to write about this for a few months, and although I know LLMs are evolving rapidly, I think it’s probably still relevant. (Let’s see ChatGPT pull off convincingly human snark.)

Earlier this year, I received an email from a client.

First, I should probably just mention that I have different types of client relationships. The kind I prefer to have is one where I’m involved with their web project from the beginning. Those clients see the true value of what I offer. (And I don’t mean “value” as in “cheap.” I mean “value” as in “worth the premium price.”)

Then there are the clients who, for whatever reason, fell into a working relationship with me after their website was already live. I’m generally reluctant to take on ongoing support for websites I didn’t build, but for various reasons, it does sometimes happen. I still avoid it when I can.

Let’s just put it this way: I have never been hired to take over support for an existing site, logged in, and thought, wow, the person who built this is really good at making websites. There’s a reason the client stopped working with them. But. If that’s the past experience the client is bringing to their relationship with me, they probably think everyone who does what I do sucks. Usually I have the opportunity to convince them otherwise, but not always. Some clients come in with an unshakeable predisposition against anyone who does what I do… especially ones who charge my rates.

As you may have guessed from those last three paragraphs, the email came from one such client. They have apparently been working with someone else (cheaper) to redesign their site, but in the meantime (going on multiple years now) they’re still stuck with me making updates to the piece of garbage I inherited. But they definitely try to keep my hours to a minimum. Which, all things considered, I get.

So, it was funny when I received this email from the client. It included a couple of attachments, both plain text (.txt) files. The client said they had created a web form they needed me to post on the site.

Well, first of all, we have Gravity Forms installed on the site, as with every WordPress site I build that needs forms. So, why didn’t they use that? I have no idea.

I opened up the text files. One was actually an HTML file. It contained their form, and some JavaScript for conditional interactivity — showing/hiding certain fields based on the selections in other fields. It was clean code and it looked like it worked. I was… surprised. (No CSS though, and obviously no page layout elements.)

Then I took a look at the other text file. It contained PHP code. It was well-structured, valid code. Technically.

But… well… I kind of just have to show it to you (with identifying details redacted, of course):

<?php
if ($_SERVER["REQUEST_METHOD"] == "POST") {
    $to = "redacted@example.com";
    $subject = "Redacted Form Submission";
    $message = "";
    
    foreach ($_POST as $key => $value) {
        $message .= ucfirst($key) . ": " . htmlspecialchars($value) . "\n";
    }
    
    $headers = "From: no-reply@example.com";
    if (mail($to, $subject, $message, $headers)) {
        echo "Email sent successfully.";
    } else {
        echo "Email failed to send.";
    }
}
?>

Although this may have been a reasonable “my first web form” tutorial for learning PHP in 2005, you can’t use this code today. Do not use this code today.

Aside from the fact that it’s not a complete page in itself — I mean, do you really want the confirmation after the user submits the form to just be “Email sent successfully” in Times New Roman, black text in the top left corner of a blank white page? Because that’s what would happen if this code ran as the response to the form submission — aside from that, this is missing so much that it would need to make it usable on a modern website.

Also set aside the fact that this is only coded to send off an email. No saving the information to a database, which you’d almost surely want on any database-driven website, especially given the current tenuousness email delivery, which I’ll get to shortly.

One safeguard is the fact that it probably wouldn’t even run successfully at all on most modern web servers, because few servers support the straight-up PHP mail() function anymore, because it’s so easy for spammers to abuse if they manage to hack into your site.

Even if the server does support the mail() function, you’ll never receive the email because, these days, any random web server that actually lets you use mail() is almost certainly already on every spam blocklist, or doesn’t have the necessary SPF, DKIM and DMARC DNS entries that receiving mail servers will check before accepting the incoming message.

Then there’s the fact that there is absolutely zero data validation or sanitization on the form input. It is trivially easy for hackers to abuse a script like this to inject arbitrary code, potentially granting them access to manipulate the contents of your database or even the server’s operating system.

Should I go on? I could go on. But I’ll leave it at that.

Here is where, Aristocrats-style, I deliver the punchline, which you’ve probably already deduced from the title of this post.

“Where,” I asked the client, “did you get this code?”

“ChatGPT.”

ChatGPT. Now, I know a lot of experienced developers these days are using LLMs to generate code, as an assistive tool to bootstrap their applications faster.

But those tools are only effective if you know how to write the correct prompts, and, critically, if you understand the code well enough that you can review it for accuracy and security before deploying it. These are not tools that are suitable for non-technical people to use to directly generate production code in 2025. Will they be someday? Probably. But we are nowhere near that point yet.

Fortunately, the client didn’t know how to install this code directly on their site, so they had to ask me for assistance with that final, crucial step. And I used the opportunity to inform them (more kindly and succinctly than here) why it was not usable as-is. It took me less than 15 minutes to replicate and test in Gravity Forms, and they were up and running with a functional, well-designed, and secure form.

So, again, please, if you don’t possess the ability to look at code and understand whether or not it will work, or what the security implications might be, don’t use ChatGPT (or any other AI) to write code.