Artifacts of an error

A little over a week ago I did something really stupid. Today I took the final step in making amends for that error, which was ultimately the consequence of a questionable decision I made many years ago, to create an ad hoc WordPress multisite setup. That lowercase “m” is significant, and I hinted at it in the post I linked to above.

This was not a proper WordPress Multisite setup. I think that feature probably already existed at the time, although it may not have. This was my own concoction, created by adding a PHP switch to the wp-config.php file, telling it to use a different database and uploads directory depending on the domain name in the HTTP request. It allowed me to manage a number of my own and Sara’s websites with a single WordPress installation, but it was dangerous.

None of the sites “knew” about each other. That wasn’t really a problem, except when it came to managing themes and plugins. I always had to be careful to remember that just because my site wasn’t using a particular theme or plugin, didn’t mean one of Sara’s sites wasn’t… or even one of my own other sites in the setup.

I forgot about that little detail when I did my stupid thing a couple weeks ago. I deleted all of the themes my blog wasn’t using, forgetting that Sara was using many of them. In a panic to quickly restore the themes, I made matters worse by accidentally restoring the entire server from a 6-day-old backup, erasing all of the work Sara and I had done on our respective sites over the preceding week.

My “final step” in making amends was to at least remove my own sites from that setup, so it is now just running Sara’s sites. If you’re reading this, it means you’re seeing my blog in its new home. (Coincidentally, this is also facilitating my gradual transition away from Digital Ocean in favor of Linode for my hosting needs.) I also moved my John Coltrane and Adelia Haight McCormick sites, which were part of the same cockamamie arrangement.

One last thing that I didn’t think I’d be able to recover: I wrote two fairly substantial blog posts in that week, and I was pretty sure those were gone forever. But last night I discovered that the Feedly app on my iPhone still had them cached! I quickly snapped a series of screenshots so they wouldn’t be lost forever.

I had originally intended to retype them here, back dating them to their original post dates, to make it seem like they’d never been lost. But somehow it seems more fitting to just post those phone screenshots instead.

One of the posts hinged on a composite Mac screenshot I had taken from Safari, demonstrating its absurd Experimental Features menu, and that’s an image I still had on my Mac, so I’m including it below as well.

Enjoy.

Also worth noting here… this is my first post on this blog using the Block Editor, a.k.a. Gutenberg. I’ve finally accepted its inevitability. I’ll probably update the site to a new theme in 2023 as well… this one is a custom hack of Twenty Eleven, for cryin’ out loud!

When WordPress Treats an Administrator Like a Contributor

The first sign that something was wrong was when I tried to create a new page on the client’s site. The blue Publish button I normally see was replaced with Submit for Review. What the…? That’s what WordPress users with the lowly Contributor role usually see. But I’m an Administrator — the most mighty role known to the world of (single-site) WordPress. (Yes, multi-site installations also confer the fearsome title of Super Admin upon a select few.)

Worse still, if I tried to click Submit for Review, it wouldn’t actually save!

Other problems abounded — I tried to create a new user with Administrator privileges, just to see if my own user account was corrupt. Couldn’t save that, either.

I had Debug Bar installed, and I noticed it was giving an error:

WARNING: wp-admin/includes/post.php:641 - Creating default object from empty value
get_default_post_to_edit

Well, that’s not good. Googling the error didn’t lead to anything immediately helpful, besides this comment that led me to explore the database structure in phpMyAdmin for any problems.

Yes, there were problems. Many of the tables, including wp_options, wp_posts, wp_postmeta and wp_users were missing their primary keys. A bit more digging into the WordPress core showed that, for complex reasons (i.e. I don’t totally get it), without primary keys on these tables, WordPress can’t determine the post type of a new post, and if it can’t determine the post type, it can’t determine the user’s capabilities with regard to that post type, which all comes back to…

WARNING: wp-admin/includes/post.php:641 - Creating default object from empty value
get_default_post_to_edit

Googling on the matter of WordPress tables missing their primary keys (or, perhaps more pertinently, their auto-increments), led me to a solution!!

Fixing WordPress indexes, foreign keys and auto_increment fields

Well, a partial solution. Because the database I was working with was not damaged in exactly the same way as the one the OP was working with, I couldn’t use the sample code directly. I had to go through the database and manually create a few primary keys, delete a bunch of auto-draft posts that all had an ID of 0, etc. Then I had to skip a few lines of the OP’s SQL code because they referred to tables that hadn’t lost their keys in my case, for whatever reason. But this is the… key… to solving the problem.

Now then, how did the database get this way? Well, the site lives on a fairly creaky old Fatcow (ugh, that name) shared hosting account, running an old version of MySQL and an almost unrecognizably ancient version of phpMyAdmin. We were undertaking major content changes on the site, so I copied it over to my own sleek, modern staging server running the latest and greatest of everything. The idea was that we’d get all of our changes in place just the way we wanted on the staging server, rather than mess up the live site for 2-3 weeks, and when we were done, we’d just copy everything back over.

Slick. Right? Sure, if both servers are running reasonably identical software versions. Which of course is never the case. Ever.

Apparently when I copied the site back to Fatcow, due to the older MySQL (or possibly phpMyAdmin) version, certain things like the primary keys and auto-increments — and, I’d be willing to bet, but I’m not sure it matters, the collation as well — got lost along the way.

Barack Obama: the open-source candidate?

Now we’re speaking close to my heart. Granted, I’m a freeloader in the open source world. I have yet to contribute a single line of code to an open source project. (OK, I guess that’s not entirely true: I did write a WordPress plugin. Sweet. I’m in the club! Sort of.) But I have wholly embraced open source software in my work. PHP FTW, baby! (Uh yeah… whatever.)

These days the only thing I’m a more enthusiastic and outspoken proponent of than open source software is Barack Obama. So I’m surprised it took me so long to research what he’s running his website on.

Linux PWS/1.3.28

*Whew* Glad to see it’s Linux. But what the heck is PWS? I was at a loss. Then I found this blog talking about the very same issue. And suddenly it made sense why I didn’t recognize the acronym. I never would have considered Microsoft’s Personal Web Server to be the web server of choice running on a Linux server. I am still scratching my head at it. The whole VM thing seems the only logical explanation, except that there’s no logic to explain it. At least it’s not so transparently ass-backwards as John McCain’s configuration:

Linux Microsoft-IIS/6.0

And the inexplicable:

Linux ECAcc (lhr)

Interestingly, though, a Google search for “ECAcc (lhr)” reveals a link to a Digg post entitled John McCain Owns VoteForTheMILF.com. Stay classy, San Diego.

Let’s be clear: I think the idea of running a web site under Windows in a virtual machine on a Linux box is the most incomprehensible, mind-bogglingly stupid arrangement you’d ever bother with. I’d have to guess that the sites were developed to run in a Windows environment, but when it came time to deal with practical server and network capacity issues, load balancing and whatnot, some sysadmin made the (probably prudent) decision to load balance on Linux boxes instead of Windows, but since the site was tied to some feeble Windows technology, they couldn’t just move it over to Linux wholesale.

But let’s take this a step further. Back in late spring I received an email from Barack Obama’s IT director soliciting applicants for web developer positions with the campaign. Even though the job was in Boston, I figured it would be insane not to apply, so I submitted my resume. (I never heard back, for what it’s worth.) And it’s from this that I happen to know that the campaign was specifically seeking PHP developers. Rock on.

With that in mind, the whole Windows-on-Linux-through-VM arrangement made even less sense. Why would they develop the site in PHP, run it on a Windows server (definitely not the optimal arrangement for a PHP-based app, though it certainly will work), and then VM that Windows environment on a Linux box, instead of just gearing the PHP app for a Linux server in the first place? And that’s when I remembered that just earlier in the day I had been looking at taxcut.barackobama.com. Of course! Separate third-level domains are all over Obama’s site. Let’s check the configuration on that domain. Now that’s much better:

Linux Apache/1.3.34 (Debian) mod_gzip/1.3.26.1a AuthMySQL/4.3.9-2 PHP/5.2.0-8+etch10

And I think it explains a lot. Campaigns start off small. Obama had to register barackobama.com and put something up there ages ago, long before he was the Democratic nominee and the hugely successful fundraiser he became along the way. So that original site, www.barackobama.com, was probably developed on a Windows box in someone’s proverbial basement, probably when was running for the U.S. Senate or maybe even the Illinois Senate. But as the campaign has grown, its websites (plural) have grown as well, and in a decidedly open-source direction. There’s some good stuff in there. Debian (which could mean Ubuntu, too… I haven’t checked the signature on Ubuntu’s Apache package to see if it’s split from its Debian roots), PHP (and a reasonably up-to-date version at that), MySQL, etc.

It’s kind of fun to do this kind of research, as long as you don’t mind being distracted along the way, because there are plenty of weird sources of distraction.

Aside from the aforementioned MILF site (classy), and the somewhat interesting fact that searching on “PWS/1.3.28” brings back as its first result a reference to Obama’s hosting, I discovered that for some reason the page title on John McCain’s official store is “Independent Online Stores.” OK. No one looks at title bars. And even fewer web developers look at <title> tags. I know that from experience. But of course that’s just a transitional landing page, announcing that McCain wares are not actually sold by the campaign, but by independent, for-profit companies, and buying these items doesn’t translate into money going back into the campaign. Huh. I can’t quite wrap my brain around that, but I’m a lifelong, union-loving Democrat, so I guess I wasn’t meant to. The only thing that comes to mind is that maybe it has to be that way, legally, now that he’s accepted public campaign financing. Anyway, the first McCain store link I found, which as they state is apparently an independent operation not affiliated with the campaign, is, not surprisingly, running:

Windows Server 2008 Microsoft-IIS/7.0

I also found that the company that hosts some of Obama’s pages also hosts a site for the American Model Yachting Association. Really? Model yachting? That exists?