How to execute a no-nonsense upgrade to PHP 7.4 on Ubuntu 16.04 LTS

Yeah, yeah. Ubuntu 16.04 LTS is getting pretty long in the tooth. Long-term support ends in less than a year.

But if you’re anything like me (I’m sorry), you’re managing multiple VPSes that are, at the moment, still running it. And now WordPress is giving all of your clients scary warnings about needing to upgrade their version of PHP. What to do?

I’ve distilled the process down to 11 lines that you can just copy-paste straight into the command line. It’s not entirely hands-off; there are a few steps where you’ll be asked to confirm whether you want to keep your existing configuration files (YES!) and such. And — very important — you’ll want to review the set of PHP-related packages I’ve got listed here to make sure they’re ones you need, and that they’re all the ones you need. If you’re not sure whether or not there are others you may want, I suggest running apt update and then apt-cache search php7.4 and reviewing the list of results before proceeding.

Now then… here we go. I’ll break it all down after the code sample.

CAVEAT EMPTOR: I’ve just run this series of commands on three servers and it seemed to work fine, but this code is provided AS IS and you’re on your own if anything gets screwed up.

This assumes you’re already in sudo mode. If not, start with a sudo -s and FEEL THE POWER.

apt update
apt -y install software-properties-common
add-apt-repository -y ppa:ondrej/php
add-apt-repository -y ppa:ondrej/apache2
apt update
apt -y dist-upgrade
apt -y autoremove
apt -y install php7.4 libapache2-mod-php7.4 php7.4-mysql php-imagick php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-gd php7.4-json php7.4-mbstring php7.4-opcache php7.4-soap php7.4-xml
a2dismod php7.0
a2enmod php7.4
service apache2 restart

OK, what are we doing here? Let’s break it down.

apt update

Updating our package cache. Gotta do this first, always.

apt -y install software-properties-common

You may already have this installed. I’m not entirely sure what it’s for but the other articles I read had me doing that before the next steps so who am I to argue?

add-apt-repository -y ppa:ondrej/php
add-apt-repository -y ppa:ondrej/apache2

We are adding external package repositories created by Ond?ej Surý that allow versions of Ubuntu Linux to install newer versions of PHP than what comes with the standard Canonical set.

apt update
apt -y dist-upgrade
apt -y autoremove

Gotta do this again, since we’ve added new repositories. We’re doing a full-blown update of any outdated packages in the OS, and using the -y switch means we’re not going to be asked to manually confirm before proceeding. Be careful!

apt -y install php7.4 libapache2-mod-php7.4 php7.4-mysql php-imagick php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-gd php7.4-json php7.4-mbstring php7.4-opcache php7.4-soap php7.4-xml

This is the big one. We’re installing PHP 7.4 as well as a bunch of related packages we probably need. If you don’t know what all of these do, I encourage you to research them. You may not need them all. You may need others not included here. But these seem to do the trick for a typical WordPress setup.

a2dismod php7.0
a2enmod php7.4

Here we’re telling Apache to stop using PHP 7.0 and to use PHP 7.4 instead. This assumes you’re currently running PHP 7.0, which would be the case if you’re still on the default Ubuntu 16.04 LTS packages.

service apache2 restart

Let’s restart Apache and get that PHP 7.4 goodness! Hopefully everything works! But I suppose we should also be forward-thinking. This command is deprecated and I believe removed completely in Ubuntu 20.04, so you could use the more modern (but to my eye, decidedly less friendly) systemctl restart apache2 instead.

Postscript

One more thing… along the way you might have updated some packages that recommend a restart. If that’s the case, throw in one last command for fun:

reboot

Obviously if your server gets a ton of traffic you may not want to reboot in the middle of the day. But then you shouldn’t have been doing any of this in the middle of the day. The Digital Ocean VPSes I use typically reboot in less than 10 seconds, so I am never too hesitant to reboot at any time. Some of the other commands above, however, may shut down Apache or MySQL for a longer period (probably not more than a minute or two).

Post-postscript

This should also work more or less the same for any other version of Ubuntu you’re trying to keep fresh past its sell-by date. The main thing you might need to look at is the a2dismod php7.0 line. You’re probably running a different version of PHP. You can use php -v to see which version you’re running, and you can run ls /etc/php to see which version(s) you have installed.

A quick fix for the impossibility of building new menus on a WordPress site with a large number of pages

I’m working on a theme migration for a WordPress site that as a lot of pages. A lot of pages. 451 pages.

The old version of the site was using page menus, not custom menus, but it was relying on a constellation of abandoned plugins, and that approach just won’t work for the new theme. We need to build custom menus.

The problem is, building a new menu on a WordPress site that has 451 pages is a daunting task. Granted, only 39 of these pages need to go into the main navigation. (A lot of them really should probably be deleted, but that’s the client’s call, not mine.) So, I’m glad I don’t have to wrangle over 400 pages into a menu, but even 39 can be difficult given the shoddy interface WordPress provides for finding pages to add to your menu.

There are three tabs: Most Recent, View All, and Search. Most Recent is useless unless you’re just adding recently created pages to an existing menu. View All is useless because it’s paginated, inside this tiny box, and it lists the pages in an entirely inscrutable order. (OK, it’s not inscrutable. I can scrute it. But just because I understand the logic of how they’re ordered doesn’t mean that order is easy to work with.) And lastly we have Search which seems like the saving grace. But it’s actually the most maddening of the three because of two things:

  1. WordPress search sucks. It doesn’t give extra weight to page titles; it searches the full content. So even if I type in the exact title of the page I want, it’s usually not first in the list. And that’s a problem because…

  2. It only returns 10 results. That’s it. Ten. No lazy loading of more, no pagination, nothing. If your page doesn’t come back in the top 10 results, it may as well not exist.

I’d really love to rebuild the internal WordPress search engine to be smarter about weighting titles. Well, OK, no I wouldn’t. That would be a project I would not enjoy. But I would like for it to be done by someone.

Since that’s not likely to happen, at least there’s a way we can modify the search results to change the number of pages returned. I found the solution, as I often do, on StackExchange. But I dislike a few of the answer’s coding conventions, and I wanted to make one specific change for myself, so here’s my version. (You may not like using closures, especially in a scenario like this because it prevents you, or anyone else, from being able to remove this logic elsewhere. If I were writing a public plugin for this, I’d definitely make it a named function, but this should be fine for a custom theme.)

add_action('pre_get_posts', function($query) {
  if (is_admin()) {
    if (isset($_POST['action']) && $_POST['action'] == 'menu-quick-search' && isset($_POST['menu-settings-column-nonce'])) {
      if (is_a($query->query_vars['walker'], 'Walker_Nav_Menu_Checklist')) {
        $query->query_vars['posts_per_page'] = -1;
      }
    }
  }
  return $query;
}, 10, 2);

The important difference between the StackExchange sample and my code is that I changed the results from 30 to -1 which, in the WordPress universe, equals . Fun!

The standard warning is that setting posts_per_page to -1 is inherently risky because it could cause performance problems. But in my testing of this change, it does not appear to be an issue on this site with 451 pages, so I’m guessing it won’t be for you, either.

Now, instead of getting back a paltry ten results, you’ll get all the results that match your search. And the exact page title you typed in should be in there, somewhere.

Add arbitrary product data to order items in WooCommerce

This seems to be way more convoluted than it needs to be, but I’m not sure how much of that is that it’s actually convoluted, how much is that Woo’s documentation sucks, and how much is that everyone else’s tutorial on it is tl;dr.

Anyway… I just wanted to do something fairly simple. I want to have each product’s short description get sent into the order data. This is a specific use case with a client who’s syncing data over the REST API with an external system, and we’re shoehorning data into the short description that maybe could go somewhere else. The point is, use your imagination as to how this might be useful to you.

I’m stripping out a lot of the other details. All I want is a way to a) add the data to the item in the cart, and b) carry that data over into the order item meta data in the database. You may need or want more, but this will get you started.

// Add custom order item meta data to cart
add_filter('woocommerce_add_cart_item_data', function($cart_item, $product_id) {
  if (!isset($cart_item['short_description'])) {
    if ($product = wc_get_product($product_id)) {
      $cart_item['short_description'] = $product->get_short_description();
    }
  }
  return $cart_item;
}, 10, 2);

// Add custom order item data from the cart into the order
add_action('woocommerce_checkout_create_order_line_item', function($item, $cart_item_key, $values, $order) {
  if  (isset($values['short_description'])) {
    $item->add_meta_data('Short Description', $values['short_description'], true);
  }
}, 10, 4);

This is a major distillation of stuff I found in these two tutorials: How to Add a Customizable Field to a WooCommerce Product and Add Custom Cart Item Data in WooCommerce.

Please, web font vendors, learn how to use CSS @font-face properly!

This has been bugging me for years and I can’t believe it’s still happening.

Being able to use custom fonts has been a huge boon to web design. And font hosting services like Typekit (sorry, I will never call it Adobe Fonts) and Google Fonts make using custom fonts easy.

But sometimes you still buy a font license that involves hosting the font files directly on your own server, and that’s where things get absolutely maddening because, for some reason, someone early on grossly misinterpreted how to use @font-face and that error has been perpetuated by countless unthinking others.

(Yes, I’m being harsh. But this is really not that complicated. And getting it right makes writing your CSS and HTML so much easier.)

Here’s an example of some font-specifying CSS you might receive from a font vendor:

@font-face {
    font-family: 'Font-Name-Regular';
    src: url('Font-Name/Font-Name-Regular.woff2') format('woff2'),
         url('Font-Name/Font-Name-Regular.woff') format('woff');
    font-weight: normal;
    }
.Font-Name-Regular {
    font-family: 'Font-Name-Regular';
    }

@font-face {
    font-family: 'Font-Name-Regular';
    src: url('Font-Name/Font-Name-Regular-Italic.woff2') format('woff2'),
         url('Font-Name/Font-Name-Regular-Italic.woff') format('woff');
    font-weight: normal;
    font-style: italic;
    }
.Font-Name-Regular-Italic {
    font-family: 'Font-Name-Regular';
    font-style: italic;
    }

@font-face {
    font-family: 'Font-Name-Bold';
    src: url('Font-Name/Font-Name-Bold.woff2') format('woff2'),
         url('Font-Name/Font-Name-Bold.woff') format('woff');
    font-weight: normal;
    }
.Font-Name-Bold {
    font-family: 'Font-Name-Bold';
    }

@font-face {
    font-family: 'Font-Name-Bold';
    src: url('Font-Name/Font-Name-Bold-Italic.woff2') format('woff2'),
         url('Font-Name/Font-Name-Bold-Italic.woff') format('woff');
    font-weight: normal;
    font-style: italic;
    }
.Font-Name-Bold-Italic {
    font-family: 'Font-Name-Bold';
    font-style: italic;
    }

This is, in fact, the exact code I just received yesterday from a font vendor when I purchased a license, with the actual font name removed to protect the guilty innocent.

What’s so bad about this, you might ask? Aside from the conventions I dislike of indenting the closing } and using 4 spaces instead of tabs, there are two glaring problems with this:

  1. Because the font-family name defined for each weight and style is different, when you go to use this font, you need to specify the font-family every time you want to use bold or italics in your HTML, or at least use the custom CSS classes defined here. No! No no no! You should not have to apply a class to get bold or italics to render properly. The <strong> and <em> tags should do that on their own!
  2. Don’t f***ing define a bold font with font-weight: normal;! If you don’t realize from this, alone, that something is wrong with your approach, stop coding right now.

So, how should this be done, you ask?

Well, it’s simple. Each @font-face declaration has four properties. One is src: which tells the browser where to find the correct font file(s) for this face. The other three properties work together to define the context in which this particular src should be used: any time this combination of font-family, font-weight and font-style come together.

You can use the same font-family in different @font-face declarations as long as font-weight and font-style are different. In fact, you’re supposed to! That’s the way it’s designed to work!!!

When you do this properly, you don’t need any custom CSS classes. Try this on for size:

@font-face {
    font-family: 'Font-Name';
    src: url('Font-Name/Font-Name-Regular.woff2') format('woff2'),
         url('Font-Name/Font-Name-Regular.woff') format('woff');
    font-weight: normal;
}

@font-face {
    font-family: 'Font-Name';
    src: url('Font-Name/Font-Name-Regular-Italic.woff2') format('woff2'),
         url('Font-Name/Font-Name-Regular-Italic.woff') format('woff');
    font-weight: normal;
    font-style: italic;
}

@font-face {
    font-family: 'Font-Name';
    src: url('Font-Name/Font-Name-Bold.woff2') format('woff2'),
         url('Font-Name/Font-Name-Bold.woff') format('woff');
    font-weight: bold;
}

@font-face {
    font-family: 'Font-Name';
    src: url('Font-Name/Font-Name-Bold-Italic.woff2') format('woff2'),
         url('Font-Name/Font-Name-Bold-Italic.woff') format('woff');
    font-weight: bold;
    font-style: italic;
}

Aside from the fact that this eliminates 1/3 of the lines of code, it also will make your HTML much cleaner and more properly separates content from styling.

Here’s an example of some HTML you might have to write using the first approach:

<p class="Font-Name-Regular">This is some regular text, which also
includes a bit of <em class="Font-Name-Regular-Italic">italics</em>
and even a dash of <strong class="Font-Name-Bold">bold</strong>.</p>

Now, granted, my version does require you to define the font-family for your <p> tags in your CSS file. But guess what… you’re supposed to do that! Put this in your CSS:

p { font-family: 'Font-Name'; }

With that in place, the proper HTML for the same appearance becomes this:

<p>This is some regular text, which also
includes a bit of <em>italics</em>
and even a dash of <strong>bold</strong>.</p>

So, again… when thinking about @font-face, just remember these two simple things:

  1. All @font-face declarations for the same font family should have the same font-family. (Seems kind of obvious when I put it that way, doesn’t it?)
  2. The value for font-weight should be the actual weight of the font. Only the “regular” weight should have font-weight: normal; or font-weight: 400;. If you’re using font-weight: normal; on a bold font, you’ve done something wrong.

This change makes for cleaner code, easier maintenance, and proper separation of content from design.

Addendum

Shortly after I posted this, I went back and looked at the unnamed font vendor’s sample page, because I knew it referenced “the @font-face standard since 2017”. I could not believe that this approach was actually a “standard,” so I tracked down the source, an article Bram Stein published on A List Apart in 2017 called Using Webfonts.

Guess what… Bram Stein’s examples do it the right way!

I do know one place where I’ve consistently seen this wrong way I’m railing against… it’s the code generated on FontSquirrel (no link, on principle) whenever you download a font. Other “web font generator” sites like FontSquirrel probably do it to. They’re all wrong… but Bram Stein isn’t. Don’t drag him down with this bad code!

I spent 5 hours troubleshooting this WordPress problem so you don’t have to (starring: WooCommerce Action Scheduler)

Sorry for that “clickbaity” headline. I added the parenthetical so it might be at least marginally useful. Since my WordPress-related posts are always about how I solved a particularly weird or obscure WP issue, I usually consider their titles carefully. “What would I have googled to find a solution to this problem?” But honestly, I spent 5 hours on this yesterday partly because I wasn’t sure what to google. (And I use lowercase “google” as a generic for “conduct an Internet search”; I normally use DuckDuckGo.)

OK, so here’s the situation. This particular site is — among my normally very-low-traffic clients — one of the busiest I work on. It’s a WooCommerce site with hundreds of products and 20+ daily orders. (Yeah, 20+ orders a day is not huge, but on the scale I normally deal with, it’s a lot.)

This site runs on its own virtual private server, with 8 GB RAM and 4 vCPUs. Pretty substantial for a single site. And yet, for weeks it has been maxing out RAM and CPU resources. Not to the point where the site was in crisis mode that demanded my immediate attention, but it was frustratingly slow. Just slightly below the threshold of me dropping work on other new projects to try to fix this. (At this point I feel obliged to note that I did not actually build or maintain this site for its first couple of years of existence, so I don’t know its inner workings as well as I normally would. I just know it’s running way too many plugins and desperately needs some TLC I have not had time to give it.)

Yesterday things finally got to the breaking point. For me, at least. The client had contacted me about an unrelated issue, but as I was dealing with that, I got frustrated by seeing all of this inexplicable resource usage, so I had to address it.

As it happens, this post is actually a bit of a sequel to my last post, about getting Apache’s mod_status and mod_rewrite to play nicely on a WordPress site. About three weeks ago I finally got mod_status working on this site, and had planned to come back, when I had a chance, to investigate this issue.

If you are not familiar with mod_status, you should check it out. Apache is generally a bit of a “black box” but this lets you see exactly what’s happening with each thread — the requested URL, the requesting IP address, connection time, resource usage, etc.

I noticed an absurd number of threads were coming from the localhost and were requesting wp-admin/admin-ajax.php with a query string referencing WooCommerce’s Action Scheduler. But what to do with that information?

I’ll admit, this is where I wasted a bunch of time in fruitless searches, because I don’t know a lot about Action Scheduler. I read a few threads on the WordPress support forums and StackOverflow that kind of danced around the problem I was having but never really got at it.

Eventually I ended up in phpMyAdmin, scrutinizing the wp_actionscheduler_actions table, and trying to figure out where all of the wc_facebook_regenerate_feed actions were coming from. I used my old favorite Search-Replace-DB to try to find any instances of “facebook” in the database. (This was an utter failure, for reasons I can’t explain. But that failure was critical to why this took me so long to resolve.)

I went to Tools > Scheduled Actions in WP admin and discovered there were over 200,000 actions, although there were only about 70,000 (only!) showing up in wp_actionscheduler_actions. Mystery!

I went to wp_actionscheduler_actions again, saw that those wc_facebook_regenerate_feed actions had all been scheduled weeks ago, and decided to just chuck out the lot. I truncated the table, but within seconds it started filling up again with hundreds of wc_facebook_regenerate_feed actions, with the same weeks-old scheduled dates. Where were they coming from???

What was especially maddening to me about all of this was that I had already, weeks ago, determined that the plugin that had created these — Facebook for WooCommerce — had been causing some kind of trouble, and I had deactivated it. Yesterday I even went so far as to delete it. I scoured the theme code for references to Facebook. I looked in the file system for stray files that might be responsible. And as I mentioned before, I tried to search the database for any references to Facebook. I was getting nowhere.

Eventually I realized Search-Replace-DB was having problems, so I dove into phpMyAdmin directly and started searching individual fields, in individual tables, for “Facebook”. And that’s where I finally figured it out.

WordPress puts everything in wp_posts, and that’s a problem.

I’ve complained over the years about the database architecture in WordPress. Having built multiple custom CMSes in the years prior to when I finally, fully embraced WordPress in 2014, I have a fair bit of experience designing databases. And two things I learned in that experience were: 1) clearly define what your data tables are for, and 2) indexes make databases efficient. WordPress is awesome for many things, but it has far, far outgrown its original conception as blogging software. Custom Post Types and Custom Fields make it super-flexible, but shoving everything they create into wp_posts and wp_postmeta can create a disastrous situation.

Case in point, WooCommerce scheduled actions. In earlier iterations, those were custom posts! (As are, still, WooCommerce orders, which is totally f***ed up, if you ask me.) At some point Woo or Automattic realized scheduled actions don’t belong in wp_posts, so they created four new tables specifically for managing them. Plugins that use scheduled actions had to create new scheduled actions for migrating the old wp_posts scheduled actions into the new tables.

And that’s where I found myself. Through some curious set of circumstances with this particular site, which probably at some point included someone other than me disabling WP-Cron to try to fix some other problem, 200,000+ scheduled actions from the Facebook for WooCommerce plugin (in the wp_posts table) got queued up for migration to the new tables. And as quickly as I was deleting them from the new tables, Action Scheduler (which runs once a minute!!!) was dutifully refilling them.

(And obviously they were never actually running… perhaps because I had deactivated the plugin? Or because they were simply timing out? Who knows? But here’s something I see as a flaw with Action Scheduler: it should check to see if the plugin that scheduled an action is currently active, and if not, purge the action immediately.)

At last here was the fix. I had to run this SQL query in phpMyAdmin. (Proceed with caution! Don’t just use this code… look in your database for exactly what is causing problems and adjust accordingly.)

DELETE FROM `wp_posts` WHERE `post_title` = 'wc_facebook_regenerate_feed';

Note: I’m doing this from memory — and a glance back at my browser history from yesterday. I didn’t keep notes on exactly what the title was.

For a more generalized — and drastic — approach, you could also do this:

DELETE FROM `wp_posts` WHERE `post_type` = 'scheduled-action';

I’ll just conclude here with a nice little graph of the site’s CPU and RAM usage over the past 24 hours. It was 6 PM when I finally figured this out!