Tag: Ubuntu

Noted for future reference: Fixing a slow-to-boot Linux server

I have a few Ubuntu Linux VPSes that were originally spun up on the then-latest-and-greatest 16.04 LTS. Over the past year I’ve been belatedly upgrading them to 20.04 LTS. Almost without exception, all now have a really irritating flaw: where they used to reboot almost instantly — making me capriciously run OS updates involving reboots at any time of day, even on servers with a bunch of sites on them, since it only meant a blip of about 5 to 7 seconds — they were now consistently taking 2 to 5 minutes to reboot. Yikes!

Poking around, I learned that cloud-init was timing out, causing that delay, but since systems administration is just a small sliver of the work I do, I never had a chance to investigate why it was happening or really what cloud-init was even for. I just resigned myself to having to do those reboots in the middle of the night when no one would notice.

Well, I finally decided I needed to get an answer, and I found it. If I’m reading this correctly, cloud-init is really only needed during the initial creation of the VPS, and can safely be disabled after that. So, let’s do it!

touch /etc/cloud/cloud-init.disabled

I’m pleased to say, it works perfectly. I ran it on a test mirror of my biggest server and it worked, so I then applied it to the live server and… capriciously rebooted it, right in the middle of the day!

Shush. It worked.

Update (August 3, 2022): Mayyyyybe it’s a bit more complicated than what I described above. I just went into another server I had previously updated to 20.04, and I just went ahead and pre-emptively ran this before an update that required reboot. After the reboot, I could not connect to the server at all, other than through Digital Ocean’s direct access console. Thank goodness for that. It did not initially occur to me that this change might be why I couldn’t connect, but after trying a few other fixes without success, I just went in and deleted this new /etc/cloud/cloud-init.disabled file and rebooted, and everything came up just fine… and without any kind of delay on boot. Weird.

How to get your upgrade from Ubuntu 16.04 LTS to 20.04 LTS to work properly

So, like me you’ve decided to procrastinate on the OS upgrade on your server running Ubuntu 16.04 LTS. Yes, support for it ended almost 6 months ago. Maybe you’ve even set up ESM (Extended Security Maintenance) to avoid the upgrade. But if you’re like me you may also have discovered that you can no longer access new PHP versions from the “ondrej” repository once LTS ends. So you reluctantly acknowledge that it would be a very good idea to do the OS upgrade.

But it doesn’t work.

That’s OK, I’m here to help. A few months ago I scraped together tips from a few different resources (which I regrettably did not make note of for citation here), and came up with a rough script for myself that seems to work. I’m running it again today on another server, so I thought I’d consolidate it into a blog post here while I’m at it. Let’s go.

Before the 16.04 to 18.04 update

First off, caveat emptor. This is what I did, and it worked. If you do the same and it breaks something, don’t blame me. Also note that I am brazen and always run sudo -s before I get started. If you don’t like to do that, you may need to prefix some of these commands with sudo in order for them to work.

It’s probably also a good idea first to do a full backup of your server that you can restore from if things go totally off the rails.

Yes, this is going to be a two-step upgrade. First we’ll upgrade from Ubuntu 16.04 LTS to 18.04 LTS. Then we can do the 18.04 LTS to 20.04 LTS upgrade to get ourselves current. Then we won’t have to think about this for another 3 1/2 years! Whew!

There’s some stuff to do before you run that do-release-upgrade command. First, edit /etc/apt/sources.list in your text editor of choice (I like nano) and add these lines at the end:

deb http://archive.ubuntu.com/ubuntu/ xenial main universe multiverse
deb http://archive.ubuntu.com/ubuntu/ xenial-security main universe multiverse

Save that. Then if you’re running ufw, which you should be, do this to give yourself a back door in case something goes wrong (assuming you’re running this update over ssh, which of course Ubuntu will warn you is a Bad Idea).

ufw allow 1022

Hold on! Failure may be looming in the near future. Before doing anything else, try this:

apt-get update
apt-get upgrade

I almost always run apt-get dist-upgrade instead of just apt-get upgrade because I always assumed the former does everything the latter does, and more. Not so. I’ve run into a problem on some servers with messages that cloud-init was held back. My fix for that is to do this:

apt-mark unhold cloud-init
apt-get upgrade
reboot

When things come back, let’s do this:

apt-get dist-update
apt-get autoremove

Then proceed as normal…

Now we’re ready to get started. Run these commands:

apt update
do-release-upgrade

Don’t walk away at this point… you’re going to have to answer some prompts along the way. In general I would always recommend keeping your existing versions of any files it asks you about. Once all of that is over, your server will reboot, and you should be able to log back into a fully functioning Ubuntu 18.04 LTS install. I would recommend testing whatever services you have running on the server, just to be sure everything is working properly, before you continue to the 20.04 LTS upgrade.

After the 16.04 to 18.04 update, before the 18.04 to 20.04 update

Run all of the regular updates, just to be sure you’re fully current. If you’re brazen like me you’ll tackle that with one command:

apt update; apt -y dist-upgrade; apt -y autoremove; reboot

Once you’re back up and running, throw this in. I don’t recall why I needed it now, and if you’re not using cURL you may not need it at all, but anyway, I did this here:

apt install libcurl4

Now, just to be safe, let’s back up the existing /etc/apt/sources.list file, because we’re going to replace its entire contents with this:

deb http://in.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse

deb http://in.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://in.archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse

deb http://archive.canonical.com/ubuntu focal partner
deb-src http://archive.canonical.com/ubuntu focal partner

(Looking just now at that in.archive.ubuntu.com domain, I’m not sure if that’s a mirror in India and you could just use archive.ubuntu.com, or what. Just an observation. Do as you will.)

Once you’ve saved those changes, run these commands:

apt update
apt install –reinstall ubuntu-keyring
do-release-upgrade

Again with the do-release-upgrade command you’ll need to follow the prompts, keeping your existing copies of config files when asked.

I’ve found that after running this, the server might not come back up on its own. I had to log into my Digital Ocean account after a few minutes and do a hard reboot, but once I did that, everything came back fine and I was on Ubuntu 20.04 LTS!

One additional note: one server I ran this on needed all of the PHP 7.4 libraries reinstalled for a reason I could not determine. Your exact setup may vary, but this is what I ran to resolve that issue:

apt install php7.4 php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-dev php7.4-gd php7.4-gmp php7.4-json php7.4-mysql php7.4-odbc php7.4-bcmath php7.4-bz2 php7.4-mbstring php7.4-soap php7.4-zip

And then I needed to edit the corresponding php.ini file with my preferred settings.

How to execute a no-nonsense upgrade to PHP 7.4 on Ubuntu 16.04 LTS

Yeah, yeah. Ubuntu 16.04 LTS is getting pretty long in the tooth. Long-term support ends in less than a year.

But if you’re anything like me (I’m sorry), you’re managing multiple VPSes that are, at the moment, still running it. And now WordPress is giving all of your clients scary warnings about needing to upgrade their version of PHP. What to do?

I’ve distilled the process down to 11 lines that you can just copy-paste straight into the command line. It’s not entirely hands-off; there are a few steps where you’ll be asked to confirm whether you want to keep your existing configuration files (YES!) and such. And — very important — you’ll want to review the set of PHP-related packages I’ve got listed here to make sure they’re ones you need, and that they’re all the ones you need. If you’re not sure whether or not there are others you may want, I suggest running apt update and then apt-cache search php7.4 and reviewing the list of results before proceeding.

Now then… here we go. I’ll break it all down after the code sample.

CAVEAT EMPTOR: I’ve just run this series of commands on three servers and it seemed to work fine, but this code is provided AS IS and you’re on your own if anything gets screwed up.

This assumes you’re already in sudo mode. If not, start with a sudo -s and FEEL THE POWER.

apt update
apt -y install software-properties-common
add-apt-repository -y ppa:ondrej/php
add-apt-repository -y ppa:ondrej/apache2
apt update
apt -y dist-upgrade
apt -y autoremove
apt -y install php7.4 libapache2-mod-php7.4 php7.4-mysql php-imagick php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-gd php7.4-json php7.4-mbstring php7.4-opcache php7.4-soap php7.4-xml
a2dismod php7.0
a2enmod php7.4
service apache2 restart

OK, what are we doing here? Let’s break it down.

apt update

Updating our package cache. Gotta do this first, always.

apt -y install software-properties-common

You may already have this installed. I’m not entirely sure what it’s for but the other articles I read had me doing that before the next steps so who am I to argue?

add-apt-repository -y ppa:ondrej/php
add-apt-repository -y ppa:ondrej/apache2

We are adding external package repositories created by Ond?ej Surý that allow versions of Ubuntu Linux to install newer versions of PHP than what comes with the standard Canonical set.

apt update
apt -y dist-upgrade
apt -y autoremove

Gotta do this again, since we’ve added new repositories. We’re doing a full-blown update of any outdated packages in the OS, and using the -y switch means we’re not going to be asked to manually confirm before proceeding. Be careful!

apt -y install php7.4 libapache2-mod-php7.4 php7.4-mysql php-imagick php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-gd php7.4-json php7.4-mbstring php7.4-opcache php7.4-soap php7.4-xml

This is the big one. We’re installing PHP 7.4 as well as a bunch of related packages we probably need. If you don’t know what all of these do, I encourage you to research them. You may not need them all. You may need others not included here. But these seem to do the trick for a typical WordPress setup.

a2dismod php7.0
a2enmod php7.4

Here we’re telling Apache to stop using PHP 7.0 and to use PHP 7.4 instead. This assumes you’re currently running PHP 7.0, which would be the case if you’re still on the default Ubuntu 16.04 LTS packages.

service apache2 restart

Let’s restart Apache and get that PHP 7.4 goodness! Hopefully everything works! But I suppose we should also be forward-thinking. This command is deprecated and I believe removed completely in Ubuntu 20.04, so you could use the more modern (but to my eye, decidedly less friendly) systemctl restart apache2 instead.

Postscript

One more thing… along the way you might have updated some packages that recommend a restart. If that’s the case, throw in one last command for fun:

reboot

Obviously if your server gets a ton of traffic you may not want to reboot in the middle of the day. But then you shouldn’t have been doing any of this in the middle of the day. The Digital Ocean VPSes I use typically reboot in less than 10 seconds, so I am never too hesitant to reboot at any time. Some of the other commands above, however, may shut down Apache or MySQL for a longer period (probably not more than a minute or two).

Post-postscript

This should also work more or less the same for any other version of Ubuntu you’re trying to keep fresh past its sell-by date. The main thing you might need to look at is the a2dismod php7.0 line. You’re probably running a different version of PHP. You can use php -v to see which version you’re running, and you can run ls /etc/php to see which version(s) you have installed.

Installing Ubuntu on an HP Stream Mini PC

When I heard the news a few weeks ago that HP was introducing a cute little blue mini PC that would only cost $180, my first thought was, “I want one!” And when I heard that the Stream Mini would ship with Windows 8.1 preinstalled, my second thought was, “And when I get it the first thing I’m going to do is install Ubuntu on it.”

Easier said than done.

The actual process of installing Ubuntu is not difficult. The difficulty proved to be in simply getting the Stream Mini to boot from a USB stick.

tl;dr

The HP Stream Mini has UEFI boot. If you hit F12 at boot, you get the UEFI boot selector screen, but that’s no good because it won’t show the USB stick. For HP devices like this you need to hit F10 on boot to access the UEFI/BIOS configuration tools. From this point I trust you can find the settings to change the boot order. Just put your USB stick ahead of the hard drive in the order and you’re set.

Actually, I do prefer to read the whole thing, no matter how long it is

Now, for those of you who want the full story, here’s how it goes. I’ll say up front that I primarily use a Mac, so my instructions for preparing the USB stick are Mac-only.

Actually, not quite true. Most of what I’m describing is done at the UNIX command line, so for a UNIX/Linux system, the experience should be similar to this, but you don’t have to convert the .iso file. Also, for the first couple of steps I’m using the Mac-only command diskutil. The instructions on the Raspberry Pi page mentioned in step 2 use the generic UNIX equivalents.

For Windows… you’re on your own. It involves crappy third-party apps you have to download and, well, yuck. I started out all of this trying to get it set up using the Windows 8.1 install that came on the HP Stream Mini, but I quickly gave up and switched to my Mac. (The official Ubuntu docs have instructions for Windows that I read before giving up.)

Step 1: Download Ubuntu

There are many options (including other Linux distros), but if you’re undecided, you probably want to get yourself a copy of the latest version (or latest LTS [Long Term Support] version) of Ubuntu Desktop. Be sure to go for the 64-bit version. At the time of this writing, Ubuntu 14.04.1 LTS is the latest and greatest.

Step 2: Prepare your USB stick

Déjà vu? No, I really have written about this before. More or less. While there are some minor differences, this is basically the same process I described for preparing an SD card with Raspbian for a Raspberry Pi using a Mac.

You can read the even lengthier details there, but basically, you want to:

1) Insert a USB stick (I recommend at least 2 GB, but who can even find one under 4 GB these days?) into a USB port on the Mac. Make sure you don’t care about losing all of the data on this USB stick, because you will.

2) Open Terminal and save yourself some extra typing in the immediate future by starting with: sudo -s

3) Type diskutil list and press Enter to see a list of volumes. Look for your USB stick and note what it says at the right under IDENTIFIER. There’s a good chance it’s going to be disk1s1 but it could be something else. You just need to know the disk1 part. Don’t worry about the s1. Remember this for later.

4) Unmount the disk. Enter this command: diskutil unmountDisk /dev/disk1 being sure to replace the 1 with whatever number you saw in the previous step.

5) Find your Ubuntu installer image. It’s probably in your Downloads folder. Type cd ~/Downloads and press Enter. Then type ls -al and look for the Ubuntu filename, which likely ends with .iso. You’re going to need to convert this into a .img file.

6) Run this command: hdiutil convert input.iso -format UDRW -o output.img but be sure to replace input.iso with the actual filename of your Ubuntu .iso file. You can also change output.img to whatever you want… or at least the output part. This is the filename for the new .img file.

7) Run ls -al again. Mac OS X probably “helpfully” stuck a .dmg extension on your .img file, so run this: mv output.img.dmg output.img replacing both instances of output as appropriate.

OK, now we are actually ready to write Ubuntu onto the USB stick. Run this command:

dd bs=1m if=output.img of=/dev/rdisk1

Be sure to use the correct name for your .img file, and the correct disk number in that rdisk1 bit. Note that the r I added stands for “raw” and it just makes the process go a bit faster.

Now, wait. It shouldn’t take too long, but it might be a minute or so. Once you get the # command prompt again, your USB stick is ready to go. You’ll probably also get a system dialogue box warning you that the disk you inserted is unreadable. Just ignore that. Remove the USB stick from your Mac and insert it into the (powered down) HP Stream Mini.

Step 3: Boot the HP Stream Mini and access the UEFI/BIOS configuration

With the USB stick in one of the HP Stream Mini’s USB ports, power on, then immediately press and hold (or, if you like, frantically tap repeatedly) the F10 key on your keyboard. This should bring up the UEFI/BIOS configuration tool.

hp01

First, go to the Storage menu and choose Boot Order. The dialogue below will appear.

hp02

I recommend disabling UEFI Boot Sources as shown here. Then under Legacy Boot Sources look for your USB stick (not shown here because I didn’t have one inserted when I took the photo). Follow the instructions in the lower right of the box to select and change the order of the boot devices. Make sure your USB stick is above Hard Drive in the list. Press F10 to confirm.

hp03

Now go to the File menu and choose Save Changes and Exit. The computer will now proceed to boot up from the USB stick.

Step 4: Run the Ubuntu installer

This is pretty straightforward. Unlike earlier Linux distributions, Ubuntu these days has a very polished installer program that should feel very familiar to anyone who’s ever installed Windows or Mac OS X on a computer before. There are several steps such as picking your language, location and keyboard layout, but just follow the on-screen instructions and wait for the process to complete, which in my experience took about 20 minutes.

After installation is complete, the system will ask you to reboot, then remove the USB stick and press Enter. Follow those instructions and in less than a minute you should be up and running with a clean install of Ubuntu!

Ubuntu hits the big time, for real this time

I just finished installing Ubuntu Linux 9.04 on my MacBook under Parallels Desktop. In the past, I’ve ventured boldly into the realm of triple-boot configurations to allow my Mac to run Mac OS X, Windows XP and Ubuntu Linux. But that involves modifying firmware and overriding the standard boot process, plus splitting your drive into 3 partitions and reciting dark incantations by the light of the full moon whilst drinking the blood of a calf slain with a silver blade.

Well OK, not that last bit. I’m not sure where that comes from (oh right, that was the part about installing Windows on the Mac). But suffice to say, while it certainly could be done, setting up triple-boot was not for the faint of heart, and once it was working, the question of whether it was all worth it loomed large. And no, I cannot think in non-clichés tonight.

When I got my new MacBook a couple months ago, I decided my days of triple boot were over. I was just going to go by the book (see what I mean?) and use Boot Camp. Well, sort of by the book. Never one to take the easy road, I wasn’t just using Boot Camp, but also running Parallels Desktop. And while you certainly can run a valid Boot Camp installation of Windows XP with an OEM license (the cheapest way to go, courtesy of Newegg.com) both directly via Boot Camp and also with Parallels, Microsoft doesn’t make it easy for you. It involves calling an automated Microsoft support line, and reciting dark incantations by the light of the full moon — I mean reciting a 48-digit number displayed on your screen, answering four simple questions asserting that you have only installed Windows on one computer and you would never, ever, ever, ever lie to Microsoft or Steve Ballmer will bite the head off a live goat and put it in your bed while you sleep, and then typing in a new 48-digit number the automated system recites back to you, while it slays a calf with a silver blade, yadda yadda.

Now where was I?

Oh yeah, Ubuntu. A new version of Ubuntu Linux is released every six months, the latest version, 9.04 (so named for being released in April 2009, get it?), having arrived on the world’s virtual doorstep earlier this week. Ubuntu releases all have clever, alliterative codenames too. This one is Jaunty Jackalope. I’ve been following Ubuntu (via the triple-boot ritual) since Gutsy Gibbon (7.10), through last year’s releases of Hardy Heron (8.04) and Intrepid Ibex (8.10), the latter of which powers this website, thanks to Slicehost.

I’ve been more and more impressed with each release of Ubuntu, as the Ubuntu development team has polished the user experience — especially the once-nightmarish installation process — and as the GNOME team has simultaneously polished the desktop software most Ubuntu users live in day-to-day. While I’m still a die-hard Mac lover, I’ll admit Microsoft has been making major improvements with its GUI design (despite other notorious issues) with Vista, and Ubuntu/GNOME has been getting better and better with each new release as well. But I really feel like 9.04, Jaunty Jackalope, has finally crossed the line where Ubuntu Linux now feels to me every bit as polished, professional, usable and pleasing as a commercial OS. The installation process is far easier, and faster, than a Windows installation, and the overall experience of the interface is clean, intuitive, and responsive.

So it occurs to me, even in this age of netbooks and Microsoft’s (and to a much lesser extent, Apple’s, at least where the Mac is concerned — iPhone is definitely their top focus these days) recent floundering, while Linux, Ubuntu in particular, is making inroads, it’s still just not going to be embraced wholeheartedly as a viable desktop alternative.

Why is that? Well, obviously Microsoft doesn’t want that to happen. Linux really much more of a threat to Microsoft than it is to Apple. Even though all three OSes can run on Macs, people as a rule just aren’t buying Macs to not run Mac OS X. I’m sure it happens but… really… why? So the major hit to commercial OS developers comes when a user buys a non-Apple computer and decides to install and use Linux (Ubuntu or otherwise) as their primary/sole OS instead of Windows. So even as Linux, and the world of free, high-quality software that comes with it, reaches maturity, and Microsoft gives us an OS that is best known as the butt of jokes (not to mention viruses and malware of all sorts), why aren’t more people switching?

Ultimately there must be some software that users are relying on for Windows that they just can’t get for Linux. It’s the same argument often leveled against Macs: “not enough software.” It’s a straw man argument. Sure there is vastly more software written for Windows than for the other OSes, but 99.9% of that Windows-only software is: a) highly specialized tools for specific industries, b) utter crap, or usually c) both. Especially when we’re talking about consumer software, whatever it is you want to do can be done just as easily on Windows, Mac or Linux. The software exists. Often between Macs and Windows it’s the same software, ported from one OS to the other or developed concurrently. With Linux it’s usually open source alternatives that are every bit as feature-rich as their commercial counterparts.

Linux has decent free options for managing photos, listening to MP3s, editing video, and all of the office tasks covered by iLife and iWork on the Mac, or by Microsoft Office and the parade of unimaginatively named Microsoft tools or OEM add-on crapware that generally comes preinstalled on Windows PCs.

Except one.

No disrespect to the developers of GIMP, but the one software program Linux absolutely needs in order to be taken seriously as a desktop OS is Photoshop. That’s it. Once Adobe stops wasting its time writing terrible custom installers and decides instead to devote those resources to porting Creative Suite to Linux, it will be all over. Windows will never go away, I’m sure, but it will be reduced to a niche OS: it will live on mainly to support legacy point-of-sale systems and industrial fabrication applications and the other arcane and ugly commercial applications that companies generally deployed in huge numbers back in the Windows NT 3.51 era and have left untouched for more than a decade.

Adobe, it’s all up to you. You owe the world a karmic debt after Bridge.