Note: I’ve copied this over from my Patreon, mainly as an SEO experiment. These days I’m generally keeping the music-related stuff on YouTube and, to a lesser extent, Patreon, but I want to see whether this post is more “google-able” here or there.

---

This probably doesn’t warrant a full YouTube video. (Then again, maybe it does, but I have a gig this afternoon and I don’t have time to make one right now!)

The big band I play in (the one that has a gig this afternoon) has “The Chicken” in our books as an encore/jam. I don’t really use the sheet music for it at all because… well, every bass player should know how to play “The Chicken.” It’s just a 12-bar blues in B♭ with a funky groove.

“The Chicken” was written by Pee Wee Ellis when he was in the James Brown band, and was famously covered by the Jaco Pastorius Big Band in the early 1980s. The main part of the song, as I said, is a straightforward blues, but the Jaco recording begins with the “Soul Intro,” a gospel-infused, slow 3/4 intro.

My sheet music has a transcription of what Jaco plays in this intro, and I’ve memorized it, but it always helps to understand the underlying form when I’m going to be playing something like this.

Here’s the part as written (re-engraved by me in MuseScore so I can overlay the chords later; note that I’ve omitted a bunch of articulation marks because they’re not relevant here):

Once you’ve worked out where to play these lines on the fingerboard, this isn’t really that difficult to play. But I find that it’s a lot easier to work out where to play the lines if I understand the underlying chord structure.

I only have two things to go by here: this written out bass part, and the recording. I checked the rest of the books in our band library. I guess there’s no piano part, because the piano player is using the guitar part, and the guitar is just tacet in the intro. There’s also no score, and of course the changes for this wouldn’t be written in the horn parts. So, I’m on my own.

First I tried working out the chords just by looking at the bass part on its own. That got me… pretty close. But then I listened to the recording and tried playing the chords on the keyboard along with the music, and I discovered I was a bit off on a couple of them. (Specifically, it was minor details like treating the opening chord as a B♭Maj7 instead of just B♭, missing a passing ♭VII, not correctly guessing that the part where I’m just playing F octaves is, in fact, an F7sus4… stuff like that.)

Anyway, once I had both examined the written bass part and played my keyboard along with the recording, I came up with the chords shown below. How do you think I did?

To summarize in text, here are the changes as I have them (using percent signs for measure repeats):

B♭ | % | D7 | % |
Gm7 | B♭7/D | C6 | % |
D♭ | % | F7 | % |
G♭6 | % | F7sus4 | % |
% | % | E♭ Gm/D Cm7 | B♭ |
E♭7 | Bb6 ||

I’m still unsure about that 6 on the C6 in bar 7. It sounds weird played on the keyboard, so it’s mainly for the benefit of the bassist.

(Of course, I only noticed after I published this that MuseScore had notated the G♭s as F#s in bars 13 and 14. They’re G♭s in the original.)

My undiagnosed ADHD brain this morning (abbreviated):

While making coffee, decided I was going to have a bagel. Got out the cream cheese and realized it was moldy. Washed out the container and went to throw it in the recycling. Discovered we didn’t have a paper bag for recycling under the sink. Went to the basement landing where we keep the paper bags. Discovered the motion sensor light wasn’t working. Replaced the bulb, but found both the bulb and the sensor were dead. Threw them both away and just installed a new bulb directly in the socket. Went back to the kitchen (surprisingly, remembering the paper bag for recycling!) and made a different breakfast since there was no cream cheese for the bagel. While eating, watched some YouTube videos. Glanced at my own YouTube channel stats, and went off on a stream of consciousness writing a script for a new YouTube video I’ll probably never make. Decided to have a second cup of coffee. While the water was heating up, noticed the stove needed to be cleaned. Took off the knobs to soak them in the sink, but found the sink drain was clogged. Took apart the sink drain to clear the clog. Realized the whole thing kind of needs to be replaced, but somehow managed to restrain myself from going to the hardware store right that instant. Cleared the clog and got things reassembled. Made my second cup of coffee. Decided I needed to document the morning here.

Still haven’t even started on my actual work for the day and it’s almost noon.

I have generally had an aversion to dark mode on devices. It’s not that I object to it aesthetically; I do think it objectively looks better most of the time. And it’s not just that I’m an old curmudgeon who wishes we were still running Mac System 7 on Motorola 68000 series processors, although I do have fond nostalgia for those times.

It’s really my eyes. And even then, it’s not that I’m an old curmudgeon with failing eyes — although my eyes have definitely deteriorated through my 40s and into my 50s.

It’s my astigmatism.

I don’t just have astigmatism; I have weird astigmatism. The axis between my two eyes is turned almost 90 degrees. I can’t get glasses with full correction because it forces the muscles in my eyes in opposing directions in such a way to give me an almost instant headache. Fortunately, I can correct it enough to see reasonably well. But I digress…

Dark mode is difficult for people with astigmatism. You know those optical illusion tricks where you stare at a dot in the center of a reversed-color image for 60 seconds, then look at a blank wall and the “burn in” in your eyes shows you the actual image? My eyes do that with white text on a dark background after about 10 seconds, to the point where I can no longer read anything on the screen.

So, I rarely use dark mode. But I know I’m in a small minority.

CSS media queries now include a prefers-color-scheme setting that lets you design your website to automatically adjust to the user’s light/dark mode preference. Combine that with CSS variables and it’s super easy to add dark mode to your website. In fact, I decided this morning to add dark mode to this site, and it only took me about 5 minutes. Choosing colors that actually look good can be a bit trickier, and you have to make judgment calls about some of your accent colors.

I decided to primarily just invert the neutral colors (white, black, grays) on my site, and I used this color inverter tool to find appropriate replacements. I didn’t change most of my accent colors, because they’re part of my brand, but I did switch to a lighter shade of blue for links, since I do already have two blues in my color palette.

I used this Dark Mode in CSS Guide for a few additional improvements, specifically using filter() to adjust the brightness and contrast on images (except my logo), as well as to tone down the intensity of the PrismJS code blocks, since — at least with the implementation I’m using — it doesn’t seem to support automatic color modes.

If you’ve got your device on dark mode, or “automatic” and it’s nighttime, you should see dark mode right now! If you’re like me though, you’ll only see it if you temporarily turn on dark mode for testing purposes. And now that I’ve done this, I’m switching back to light.

I stand motionless
Staring into the hot orange glow
Tick… tick…
Seconds become minutes
Become years
Tick… tick…
The hot orange burns my eyes
As I feel death's icy grip upon my shoulder
Tick… tick…
Ding!
My toast is ready

---

I’ve been having some issues with a particular server lately where it keeps going down. I probably should have given it serious attention sooner, but it’s a “personal” server (runs this site, my wife’s blogs, and a few other sites I’m hosting as a courtesy to friends), and I’ve had a lot going on lately.

This morning it seemed to be worse off. MySQL just wouldn’t start. My monitoring script that fires off every 10 minutes so I don’t have to be on-call 24/7 was doing its best, but it just kept restarting in vain.

Time to look into the issue. I found that MySQL was running, consuming over 100% CPU (it’s a multi-CPU machine so the maximum percentage is over 100), but nothing was loading.

Running systemctl status mysql.service showed this, which kind of surprised me:

Status: "Server startup in progress"

So, something was causing MySQL to just get stuck in the startup process and never actually get up and running. I figure that is usually a corrupt database, which could be a nightmare, especially since I’ve been ignoring this issue for a week or so. Having to restore from a week-or-more-old backup would be a minor inconvenience to me, but my wife writes a lot on her blog and she would not be happy to lose several days of work.

I needed to check out the MySQL log file. At 17 GB — yikes — that meant using tail to just check out the last few hundred lines.

Here’s something interesting:

2025-06-08T14:33:14.651332Z 1
[ERROR] [MY-011899] [InnoDB] [FATAL] Unable to read page [page id:
space=0, page number=5] into the buffer pool after 100 attempts. The
most probable cause of this error may be that the table has been
corrupted. Or, the table was compressed with with an algorithm that is
not supported by this instance. If it is not a decompress failure, you
can try to fix this problem by using innodb_force_recovery. Please see
http://dev.mysql.com/doc/refman/8.0/en/ for more details. Aborting…
2025-06-08T14:33:14.651347Z 1 [ERROR] [MY-013183] [InnoDB] Assertion
failure: buf0buf.cc:4110:ib::fatal triggered thread 140583111841344
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

I went straight to the last link to the MySQL docs, Forcing InnoDB Recovery. Since this site has a bunch of WordPress databases that all use InnoDB tables, that would — hopefully — be the solution.

It’s pretty simple, once you find the right configuration file to put this line of code in:

innodb_force_recovery = 1

My server is running Ubuntu, so the file I wanted (I had to hunt around a bit) was:

/etc/mysql/mysql.conf.d/mysqld.cnf

I added that line at the end of the file, ran systemctl start mysql and much to my surprise, after about 3 seconds, the command prompt returned, with no errors. I fired up Safari and checked out my site and… well, since I’m writing this here, you can guess the rest.

Of course, is this really a solution? I was hoping so. The name of the parameter sounds like it’s, y’know, going to fix any problems it encounters. But reading the documentation further, it looks like it is really designed just to bypass certain safety mechanisms in order to allow the system to run so you can do your own troubleshooting.

Unfortunately I’m not quite sure where to begin with this troubleshooting. There are over 30 databases on this server, so I’m looking at somewhere over 500 tables, any of which could be the culprit, and the log files don’t give any indication of which table — or even which database — is the source of the problem.

So, when in doubt, I like to start as simple as possible. Since innodb_force_recovery is supposed to be only a temporary setting and it limits certain functionality, I knew I would eventually have to turn it off again. Let’s just try that now and see what happens.

I commented out the line I had just added to the config file, tried restarting MySQL, and… it worked. I’m not sure if starting up with innodb_force_recovery did do something that cleaned up the problem, or if just using that setting to get past whatever was hanging things up before allowed the normal boot process to do some standard cleanup, but in any case, it seems to be working fine now.

But if I get another alert that things have gone down, I’m not going to wait a week to investigate this time, no matter how much more pressing work I have going on.

---

Update (June 12, 2025): Although the server hasn’t completely crashed again, I’ve been observing over the past couple of days that my monitoring script is still restarting services a couple of times a day, so there’s still a problem.

I realized that the most likely database to be having issues was the one running this site, since it was originally installed with WordPress 2.1 back in 2007, and the core tables were still running the MyISAM engine (instead of the modern standard InnoDB), and several tables also had odd character set collation settings. Those have been causing problems in recent months, such as the odd issue I had with a previous WordPress update where it was mangling image uploads. (Yes, that actually had to do with the data table structure!)

I decided to rebuild the database: the tl;dr is that I exported the data to my computer, dropped all of the tables in the live database, and then re-imported the data to create clean new tables.

The long version is, I did a bit more than that. Those old tables had several NOT NULL datetime fields where the default value was 0000-00-00 00:00:00 — a value that was acceptable in some earlier version of MySQL but now is not. (The dates have to map to a valid Unix timestamp, so the new default is 1970-01-01 00:00:00. Although I went with 2000-01-01 00:00:00 as my new defaults.)

I also took this opportunity to change all of the tables to using the InnoDB engine and utf8mb4_unicode_520_ci as the collation. (Yes, I probably could have/should have used utf8mb4_0900_ai_ci but I went with the newest value I saw in other tables.)

I’ll need to give it a day or two to see if this resolves the issue. If not, I think it will mean that the principle is correct, but I just haven’t found the damaged database yet. So, I just have about 29 more to test after this!)

---

Update (August 2, 2025, Ron Howard voiceover): It did not resolve the issue.

Well, it did for a while, but the same problem happened again two months later. So I haven’t really pinpointed the cause yet. I’m inclined to just encourage the client to ditch their VPS and move to managed hosting elsewhere.

---

Update (August 4, 2025): Now that it’s Monday and I’m no longer just in “weekend emergency support” mode, I took a fresh look at this. I’m not a database administration expert, but I think my folly the first time around was that I didn’t actually do anything after getting things back up and running. The MySQL documentation emphasizes using innodb_force_recovery to get the database running so you can dump the tables. I am hoping that just dumping the tables and then restoring from the dump will clear up whatever ticking time bomb was lurking in the database. Fortunately that’s super easy with a couple of shell commands:

mysqldump -u USERNAME -p DATABASE_NAME > DATABASE_NAME.sql
mysql -u USERNAME -p DATABASE_NAME < DATABASE_NAME.sql

The first (mysqldump) command exports the database contents to a .sql file. The second (mysql) command uses that .sql file to overwrite the contents of the database. Someone please correct me if there's something more you have to do to get the data tables to be rewritten fresh from the file, but I believe that's it. (Running these two commands freed up around 200 MB of space on the disk, so it obviously did something significant.)