28 September

How to get your upgrade from Ubuntu 16.04 LTS to 20.04 LTS to work properly

So, like me you’ve decided to procrastinate on the OS upgrade on your server running Ubuntu 16.04 LTS. Yes, support for it ended almost 6 months ago. Maybe you’ve even set up ESM (Extended Security Maintenance) to avoid the upgrade. But if you’re like me you may also have discovered that you can no longer access new PHP versions from the “ondrej” repository once LTS ends. So you reluctantly acknowledge that it would be a very good idea to do the OS upgrade.

But it doesn’t work.

That’s OK, I’m here to help. A few months ago I scraped together tips from a few different resources (which I regrettably did not make note of for citation here), and came up with a rough script for myself that seems to work. I’m running it again today on another server, so I thought I’d consolidate it into a blog post here while I’m at it. Let’s go.

Before the 16.04 to 18.04 update

First off, caveat emptor. This is what I did, and it worked. If you do the same and it breaks something, don’t blame me. Also note that I am brazen and always run sudo -s before I get started. If you don’t like to do that, you may need to prefix some of these commands with sudo in order for them to work.

It’s probably also a good idea first to do a full backup of your server that you can restore from if things go totally off the rails.

Yes, this is going to be a two-step upgrade. First we’ll upgrade from Ubuntu 16.04 LTS to 18.04 LTS. Then we can do the 18.04 LTS to 20.04 LTS upgrade to get ourselves current. Then we won’t have to think about this for another 3 1/2 years! Whew!

There’s some stuff to do before you run that do-release-upgrade command. First, edit /etc/apt/sources.list in your text editor of choice (I like nano) and add these lines at the end:

deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse
deb http://archive.ubuntu.com/ubuntu/ xenial-security main universe multiverse

Save that. Then if you’re running ufw, which you should be, do this to give yourself a back door in case something goes wrong (assuming you’re running this update over ssh, which of course Ubuntu will warn you is a Bad Idea).

ufw allow 1022

Hold on! Failure may be looming in the near future. Before doing anything else, try this:

apt-get update
apt-get upgrade

I almost always run apt-get dist-upgrade instead of just apt-get upgrade because I always assumed the former does everything the latter does, and more. Not so. I’ve run into a problem on some servers with messages that cloud-init was held back. My fix for that is to do this:

apt-mark unhold cloud-init
apt-get upgrade
reboot

When things come back, let’s do this:

apt-get dist-update
apt-get autoremove

Then proceed as normal…

Now we’re ready to get started. Run these commands:

apt update
do-release-upgrade

Don’t walk away at this point… you’re going to have to answer some prompts along the way. In general I would always recommend keeping your existing versions of any files it asks you about. Once all of that is over, your server will reboot, and you should be able to log back into a fully functioning Ubuntu 18.04 LTS install. I would recommend testing whatever services you have running on the server, just to be sure everything is working properly, before you continue to the 20.04 LTS upgrade.

After the 16.04 to 18.04 update, before the 18.04 to 20.04 update

Run all of the regular updates, just to be sure you’re fully current. If you’re brazen like me you’ll tackle that with one command:

apt update; apt -y dist-upgrade; apt -y autoremove; reboot

Once you’re back up and running, throw this in. I don’t recall why I needed it now, and if you’re not using cURL you may not need it at all, but anyway, I did this here:

apt install libcurl4

Now, just to be safe, let’s back up the existing /etc/apt/sources.list file, because we’re going to replace its entire contents with this:

deb http://in.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse

deb http://archive.canonical.com/ubuntu focal partner
deb-src http://archive.canonical.com/ubuntu focal partner

(Looking just now at that in.archive.ubuntu.com domain, I’m not sure if that’s a mirror in India and you could just use archive.ubuntu.com, or what. Just an observation. Do as you will.)

Once you’ve saved those changes, run these commands:

apt update
apt install –reinstall ubuntu-keyring
do-release-upgrade

Again with the do-release-upgrade command you’ll need to follow the prompts, keeping your existing copies of config files when asked.

I’ve found that after running this, the server might not come back up on its own. I had to log into my Digital Ocean account after a few minutes and do a hard reboot, but once I did that, everything came back fine and I was on Ubuntu 20.04 LTS!

One additional note: one server I ran this on needed all of the PHP 7.4 libraries reinstalled for a reason I could not determine. Your exact setup may vary, but this is what I ran to resolve that issue:

apt install php7.4 php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-dev php7.4-gd php7.4-gmp php7.4-json php7.4-mysql php7.4-odbc php7.4-bcmath php7.4-bz2 php7.4-mbstring php7.4-soap php7.4-zip

And then I needed to edit the corresponding php.ini file with my preferred settings.

WordPress tip: A simple way to search all post types

I love WordPress, but its huge designer/developer community and extensible structure have made it possible to over-engineer a solution to just about every problem. And then under-document that solution.

Case in point: today I needed to add the ability to search across custom post types, along with pages. But by default search only searches posts. (That is, the “post” post type. Are you with me?)

This isn’t a new problem, even to me, although very few of the sites I build have (or need) internal search. It’s just not that useful on a site that doesn’t have hundreds of pages or posts, and most of the sites I build don’t.

In the few times in the past when I needed to be able to search across other post types, or other content like taxonomy data, I’ve relied on the Search Everything plugin. And judging by the fact that (as of today) it’s been downloaded 555,309 times, clearly I am not alone.

It’s a pretty good plugin, as plugins go. But it can be overkill, especially if all you need is the ability to search other post types.

And that’s where we run into the real, multifaceted problem with WordPress for developers: 1) there’s a plugin (no, make that dozens of plugins) for just about every obscure task, and 2) there are also several ways to go about building your own custom solution, especially if you’re building your own theme, but 3) the documentation is all over the place, and none of it is comprehensive.

Granted, by offering a targeted solution to a very specific problem in this blog post, I’m contributing to that documentation fragmentation, but whaddayagonnado.

There’s a fourth (and probably even more important) facet as well: plugins are developed independently by countless individuals (of varying degrees of skill), and it’s impossible for anyone to test them all for interoperability. The more plugins you install — especially if they’re excessively complex for the problem you’re intending to solve — the greater the chance you’ll introduce an incompatibility that will break your site. So it’s in your best interest to try to keep things as simple as possible. (And to err on the side of installing fewer plugins.)

tl;dr

If I understand correctly, every parameter in WP_Query can be passed in the query string, which means you can add corresponding input fields into searchform.php in your theme to modify the search functionality.

OK, now that was too simple (and abstract). Give me an example I can work with.

Here’s a one-line solution to get WordPress to search all of your post types (even custom post types), not just “post”-type posts. Add this into the form in searchform.php:

<input type="hidden" name="post_type" value="any" />

Guess what… you can also specify multiple specific types of posts using PHP’s method of using square brackets in your input name to pass in data as an array, like this:

<input type="hidden" name="post_type[]" value="foo" />
<input type="hidden" name="post_type[]" value="bar" />
<input type="hidden" name="post_type[]" value="baz" />

A caveat: I tried the above with page as one of the values and it didn’t work; it showed my custom post types, but not “page”-type posts. I suspect it’s because that’s one of the predefined type parameters that make the query behave slightly differently. So this solution isn’t perfect, but using any as the value will work: it gets “post”, “page” and your custom post types.

My goal was to have a simple search form that would just search all post types, so I made this a hidden field, but you could make it radio buttons, checkboxes or a select menu if you wanted to let the user pick, and this just scratches the surface of what you can do to customize your search form to leverage the capabilities of WP_Query.

Three

This is my latest album. It is called Three. I’ve just made it available for free streaming and download over on my music site.

Warning: here there be dragons. Well, not really dragons so much. I would classify the album as progressive rock, but not the wizards and sorcery kind of prog rock. Still, if you’re not in the mood for 20-minute rock suites or free-form improvisation, it may not be your bag, baby. There’s Mellotron. A lot of Mellotron. Never fear, there’s also a long essay describing the album’s creation in copious detail for your insomnia-curing pleasure. Enjoy!

(Note: CDs are on their way in the next week or so…)

My Favorite Pomplamoose

I was delighted by what Pomplamoose did with their cover of “Single Ladies” (mentioned in my previous post). But I am in utter, slack-jawed awe at their distinctively twisted cover of “My Favorite Things.”

This is a song I know well, having devoted my undergraduate thesis to the evolution of John Coltrane’s treatment of it over the years. Pomplamoose has taken the song in a completely different, but no less unexpected direction. It’s absolutely wonderful. And now I need to go cry in the shower, Tobias Fünke style, because I will never in my wildest dreams be able to produce something this amazing in my own home music studio.