How to get your upgrade from Ubuntu 16.04 LTS to 20.04 LTS to work properly

So, like me you’ve decided to procrastinate on the OS upgrade on your server running Ubuntu 16.04 LTS. Yes, support for it ended almost 6 months ago. Maybe you’ve even set up ESM (Extended Security Maintenance) to avoid the upgrade. But if you’re like me you may also have discovered that you can no longer access new PHP versions from the “ondrej” repository once LTS ends. So you reluctantly acknowledge that it would be a very good idea to do the OS upgrade.

But it doesn’t work.

That’s OK, I’m here to help. A few months ago I scraped together tips from a few different resources (which I regrettably did not make note of for citation here), and came up with a rough script for myself that seems to work. I’m running it again today on another server, so I thought I’d consolidate it into a blog post here while I’m at it. Let’s go.

Before the 16.04 to 18.04 update

First off, caveat emptor. This is what I did, and it worked. If you do the same and it breaks something, don’t blame me. Also note that I am brazen and always run sudo -s before I get started. If you don’t like to do that, you may need to prefix some of these commands with sudo in order for them to work.

It’s probably also a good idea first to do a full backup of your server that you can restore from if things go totally off the rails.

Yes, this is going to be a two-step upgrade. First we’ll upgrade from Ubuntu 16.04 LTS to 18.04 LTS. Then we can do the 18.04 LTS to 20.04 LTS upgrade to get ourselves current. Then we won’t have to think about this for another 3 1/2 years! Whew!

There’s some stuff to do before you run that do-release-upgrade command. First, edit /etc/apt/sources.list in your text editor of choice (I like nano) and add these lines at the end:

deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse
deb http://archive.ubuntu.com/ubuntu/ xenial-security main universe multiverse

Save that. Then if you’re running ufw, which you should be, do this to give yourself a back door in case something goes wrong (assuming you’re running this update over ssh, which of course Ubuntu will warn you is a Bad Idea).

ufw allow 1022


Hold on! Failure may be looming in the near future. Before doing anything else, try this:

apt-get update
apt-get upgrade

I almost always run apt-get dist-upgrade instead of just apt-get upgrade because I always assumed the former does everything the latter does, and more. Not so. I’ve run into a problem on some servers with messages that cloud-init was held back. My fix for that is to do this:

apt-mark unhold cloud-init
apt-get upgrade
reboot

When things come back, let’s do this:

apt-get dist-update
apt-get autoremove

Then proceed as normal…


Now we’re ready to get started. Run these commands:

apt update
do-release-upgrade

Don’t walk away at this point… you’re going to have to answer some prompts along the way. In general I would always recommend keeping your existing versions of any files it asks you about. Once all of that is over, your server will reboot, and you should be able to log back into a fully functioning Ubuntu 18.04 LTS install. I would recommend testing whatever services you have running on the server, just to be sure everything is working properly, before you continue to the 20.04 LTS upgrade.

After the 16.04 to 18.04 update, before the 18.04 to 20.04 update

Run all of the regular updates, just to be sure you’re fully current. If you’re brazen like me you’ll tackle that with one command:

apt update; apt -y dist-upgrade; apt -y autoremove; reboot

Once you’re back up and running, throw this in. I don’t recall why I needed it now, and if you’re not using cURL you may not need it at all, but anyway, I did this here:

apt install libcurl4

Now, just to be safe, let’s back up the existing /etc/apt/sources.list file, because we’re going to replace its entire contents with this:

deb http://in.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse

deb http://archive.canonical.com/ubuntu focal partner
deb-src http://archive.canonical.com/ubuntu focal partner

(Looking just now at that in.archive.ubuntu.com domain, I’m not sure if that’s a mirror in India and you could just use archive.ubuntu.com, or what. Just an observation. Do as you will.)

Once you’ve saved those changes, run these commands:

apt update
apt install –reinstall ubuntu-keyring
do-release-upgrade

Again with the do-release-upgrade command you’ll need to follow the prompts, keeping your existing copies of config files when asked.

I’ve found that after running this, the server might not come back up on its own. I had to log into my Digital Ocean account after a few minutes and do a hard reboot, but once I did that, everything came back fine and I was on Ubuntu 20.04 LTS!

One additional note: one server I ran this on needed all of the PHP 7.4 libraries reinstalled for a reason I could not determine. Your exact setup may vary, but this is what I ran to resolve that issue:

apt install php7.4 php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-dev php7.4-gd php7.4-gmp php7.4-json php7.4-mysql php7.4-odbc php7.4-bcmath php7.4-bz2 php7.4-mbstring php7.4-soap php7.4-zip

And then I needed to edit the corresponding php.ini file with my preferred settings.

How did I not know about ClassicPress before now?

ClassicPressI’ve been using WordPress for 15 years, and have made it my go-to platform for all new websites I’ve built since 2014. So how is it that it took me three years to discover that ClassicPress exists, especially since its whole raison d’être is to keep the pre-Gutenberg dream of WordPress alive?

On one hand, being a solo developer — even before the pandemic — has always kept me a bit out-of-the-loop, especially since I don’t attend conferences. But I suspect the fact that I knew nothing of this also speaks negatively to the project’s future.

Is it gaining enough traction to continue to exist? Is it really a viable option to use on new professional projects in 2021?

Has the Gutenberg ship sailed? Well, yes, it has. But my issues with the current and future state of WordPress go beyond Gutenberg, to the nature of Automattic’s role in steering the ship, the greater vision of what WordPress is and should become, and… well… Matt Mullenweg’s personality. I feel like the future of WordPress is increasingly diverging from what I hoped to get out of it as a platform, and it’s clear that I’m not alone. That’s why ClassicPress exists.

There are a lot of things to like about ClassicPress, right out of the gate, besides the most obvious element, which is the absence of anything Gutenberg. It does away with a lot of the cutesy crap that’s rolled into WordPress by default, not least of which being the annoying proliferation of the word “howdy” and the beyond-pointless-to-actively-detrimental* plugin Hello Dolly.

As I look to my own future with WordPress and/or ClassicPress, I am primarily thinking about two things: 1) how/if I will continue to use it as the platform of choice for client projects, and 2) what the future will be for the plugins I have contributed to the WordPress community, and more specifically, my commercial plugin, ICS Calendar Pro.

I’ve been struggling with these matters for almost four years now, ever since Gutenberg emerged on the scene and went through its early phases of absolutely sucking, to its too-soon release as the default WordPress editor, to its current state as a mostly good but highly quirky and weirdly limited page building tool.

The timing was not great for me, as I had just recently gone “all-in” on 34 Blocks, my own block-based starter theme that I have been using to create all of my client sites since 2017. It started from a series of one-off client themes beginning around 2015 and is built around Advanced Custom Fields and its “Flexible Content” fields. It’s all much more in line with what “WordPress” has always meant to me. But as WordPress becomes Gutenberg, my vision of what this tool is and the reality of what it has become are increasingly at odds.

In those four years I’ve been bouncing around between several different ideas:

  • Suck it up and finally embrace Gutenberg development, learning a bunch of new stuff like React, in which I am not only wholly disinterested but with which I philosophically disagree?
  • Cling for dear life to Classic Editor and pray the gods of Automattic keep it on life support?
  • Switch to an entirely new platform, whether that might be another open source or commercial CMS, or a complete SaaS approach like Squarespace?
  • Get out of the web development business entirely?

So far, I’ve mostly stuck with “cling for dear life to Classic Editor” although I have been tempted a great many times to “get out entirely.” My enthusiasm for this field hasn’t been helped by things like the caustic toxicity of social media, the rise of absolutely godawful and not-at-all-intuitive-regardless-of-their-claims-to-such interface concepts (see: Google’s Material Design), and technical snafus like Digital Ocean’s entire subnet getting spam blacklisted and them doing absolutely zero to rectify the situation.

I’ve been taking baby steps towards making sure I’m not caught out when/if they pull the plug on Classic Editor. My 34 Blocks theme is to the point where it works adequately in the Gutenberg environment, and I’m even moving it towards a potential future where I would scrap my ACF Flexible Content blocks altogether, in favor of Gutenberg blocks.

But I’ve also made sure ICS Calendar is backward-compatible with WordPress 4.9, so it works with ClassicPress. And I’m still looking at other tools, now and again, in case I need to switch directions entirely.

It’s happened before. After the first half of my career consisted largely of building “bespoke” CMSes for corporate overlords, I went out on my own. From 2008 to 2014 I sunk thousands of hours into the development of a feature-rich, completely custom-built CMS based on the CakePHP framework, which I used to create about 10 client sites per year throughout that period.

But the writing was on the wall for that project when I found it impractical to upgrade the CakePHP core past version 1.3, which was incompatible with PHP 7. (CakePHP is currently up to version 4.1 and now requires a minimum of PHP 7.2, for an indication of just how doomed my old CMS project was.) By 2014 I gave up on it and switched to WordPress. Has the time come to move on again? If so, I feel like in some ways, switching to ClassicPress would be a step backwards, or at best a lateral move, and would not set me up well for the future.

Where does that leave me? I don’t know. There are options. But embracing Gutenberg and the future of WordPress is not at the top of the list. If anything, it’s never been lower.

* Why is Hello Dolly detrimental? The justification for its inclusion in the default WordPress build is that it is a demo for new developers to learn how to build a WordPress plugin. The problem is, it’s a terrible, no good, entirely wrong example of a plugin. It’s ancient and doesn’t conform to any modern WordPress coding standards, and it’s so rudimentary that there’s no useful structure to build on for people who want to create an actually useful plugin. So why is it still included? I don’t buy the “demo” argument. It’s still there because Matt wants it to be, and that in a nutshell is my problem with Automattic running the show. (I mean look, he even “cleverly” misspelled the company name so his own fucking name is embedded in it. That annoys me every damn time I see it… almost as much as “howdy.”)