Bracing for the HD web

Joshua Johnson has an excellent new post over on Design Shack called Ready or Not, Here Comes HD Web Design, discussing adapting your web design techniques for the imminent HD revolution, being led by the new iPad.

I’ve been adapting my designs for the Retina Display on the iPhone 4/4S for a while now, but it’s easy enough to build a responsive web design that just shrinks down large desktop website images to half their size for display on a tiny iPhone screen. Making an essentially desktop-sized display support high-resolution graphics is a whole other story, and even though I knew (or presumed, along with the rest of the world) that a Retina iPad was coming, I think some small part of me was still in denial about what it would mean for web design.

Well, that future is here. Sure, most of the world is not browsing the web on a Retina Display iPad. But if you think the HD revolution ends there, think again. It’s just getting started. I’m about to enter the implementation phase on a handful of big web projects over the next month. Accommodating the new iPad’s display is going to be one of my top tasks.

I’ve been thinking for the past month or so that I was going to need to address this, and I’ve made the decision that, moving forward, on any site that has responsive web design as a component (which will probably be all sites I do), full-size “Retina Display” graphics are essential.

I posted a comment on Joshua Johnson’s post, where I noted that “Retina” (or, to avoid using Apple’s marketing term, “HD web”) graphics don’t really need to be 326 pixels per inch (ppi), the iPhone 4/4S’s resolution, or even 264 ppi, the new iPad’s resolution.

The standard practice with web images has always been to render them at 72 ppi, but the fact is, browsers don’t care what resolution an image is set to. A pixel is a pixel, and web images get displayed at the screen’s native resolution (unless, of course, you resize the image dynamically in the browser using HTML dimension attributes or CSS). The Retina Display approach Apple uses is to simply double the resolution (quadrupling the number of pixels). Web images by default get “pixel-doubled” in this scenario, displaying at the same relative size as they would if the iPad still had a low-res display, but appearing pixilated or blurry as a result.

You don’t need to render your images at 264 ppi for display on the new iPad. In fact, you can leave the resolution at 72 ppi, because the browser still doesn’t care. (Well, maybe it does; I haven’t actually had an opportunity to test it, but I strongly suspect the answer is no.) You just need to make the pixel dimensions of the image twice what they were before. In fact, even if you do change the resolution, you still need to double the dimensions. That’s the key.

After mulling all of this over for the past week, I’ve decided I’m going to have to get a new iPad (what a hardship, I know). I could technically do this work without it, but I think it’s important to be able to see what I’m producing, if for no other reason than to remind myself how important it is.

This is going to be an ongoing process, and because web design (or, more accurately, web design implementation — I usually work with designers who produce the initial UI in Illustrator or Photoshop) is only one part of my job, I can’t give it my undivided attention. But it’s something I am preparing now to immerse myself in as fully as possible. From now on, this is just how I do things. And with that in mind, there are some key points to consider:

1. Some images are more important to represent high-res than others. Logos, absolutely, 100%. Design elements that are on all pages (backgrounds, borders, navigation) come next. One-off photos are not as critical, but probably still are if they’re on the home page. But consider the next point as well.

2. Bandwidth is a concern. I’ve been somewhat dismissive of this up to now, as I’ve been focusing on high-res logos for the iPhone’s Retina Display — it can just take the regular web images and show them at half size to achieve high-res — but if you’re suddenly talking about downloading 4 or 5 high-res photos for a home page slideshow, it’s going to be a problem. Making users with standard resolution download unnecessarily large images is bad; making iPad users eat up their 4G data plan downloading your perfect-looking photos is bad too. Most of the attention paid to bandwidth in this discussion that I’ve seen so far has focused on the former, but both need to be addressed.

3. CSS and SVG. Joshua Johnson talks about this in his post. If we can render elements using CSS instead of images (things like curved corners, gradients, shadows, etc.), we should do that. More complex vector graphics can be displayed in SVG. All modern browsers now (finally) support SVG. Up to now it’s been a cool but essentially useless technology, due to lack of widespread support. But IE8 and earlier don’t support SVG, so if those browsers matter (and unfortunately they probably still do), you need a backup plan.

It is an exciting time to be working in web design and development. More and more, the challenges center around adapting your techniques to take advantage of cool new features and capabilities, not accommodating the limitations of crap browsers. But they’re still challenges, and we’re just beginning to discover their depth, along with their solutions.

Update: Over at Daring Fireball, John Gruber just linked to Duncan Davidson’s post detailing an issue with WebKit (the rendering engine inside Safari) automatically scaling down images above a certain size, specifically over 2 megapixels. Looks like sending huge, high-res images to the new iPad might present even more challenges than the ones I’d been considering.

I still don’t have that new iPad, so we’ll see what happens when I get it. (Tomorrow?)

Update #2 (March 25, 2012): A few more days have passed, and more has been learned on this topic. Duncan Davidson has a follow-up post that further explores the issue and a tentative path forward.

P.S. I did get that new iPad, and Duncan’s demo of an 1800-pixel JPEG on the iPad’s Retina Display is truly impressive. But what I find really interesting about it is that the 1800-pixel version of the photo looks better than the 900-pixel version even on a regular computer display… because, as I’ve observed, WebKit downscales images better than Photoshop does.

Vexed by OpenX: the case of the missing statistics

I am not a huge fan of online ads. But I recognize they are a necessary part of some business models, including those of some of my clients. As such, it is my obligation to offer — and support — a solution.

Knowing what I now know, I would not be likely to recommend OpenX to any client. In 2008, however, I was new to the world of ad server software, and took someone else’s recommendation to go with OpenX. Much has changed since 2008, but — for better or worse, and I’m leaning toward worse, mainly because I’ve come to realize how bad OpenX is, and that there’s probably not any better alternative — that client is still relying on OpenX for their web ads.

The intervening years have brought a handful of challenges where OpenX is concerned. Besides its excessive complexity (143 data tables! most of which by this point either contain 0 records or over 2 million!) and inscrutable structure (why are statistics a “plugin”? and why are the built-in plugins not actually installed automatically?), it is notoriously susceptible to security exploits.

Such was the case about a year ago, the first time I ran into the problem of missing statistics, when I ran the version 2.8.8 upgrade to patch a hacked installation.

This time around, I had only myself to blame. An errant maintenance script in my own CMS had inadvertently deleted all of the files in the openx directory, effectively wiping out OpenX, though thankfully leaving the precious data tables untouched. I had an outdated backup that contained most of the ad assets, and the client was able to provide replacements for the newer ones, so within the same day we were back up and running, serving ads on the website again.

Except… no statistics.

My second mistake was that I hadn’t made notes of the solution I had eventually discovered the first time we ran into this problem, so I was forced to spend several days once again trying in vain to hunt down the solution on Google. Today I finally did, so I would like to share it here, with the deepest gratitude to its author, Frederik Schøning.

Before I detail precisely what did, at long last, restore statistics to our OpenX installation, I want to describe a few of the things I did before that, which did not solve the problem, but which may have been critical to my ultimate success. (I’ve learned over the years that, when debugging, one problem may obscure another, and that there’s rarely only one thing wrong.)

First, installation itself. For quite a while, OpenX required you to jump through a number of frustrating hoops just to get the actual installer. Recently, they’ve cut the crap and provided a direct download link. I ran the installation, snagging briefly on the fact that my data tables already existed. The installer sort of accounts for this situation, however, allowing you to upgrade the existing data tables instead of creating a brand new database. (This, I believe, is where my problems with statistics began. More on that in a minute.)

After installation, I went through several iterations of repairing data tables, fixing file permissions, copying files from my old installation (be sure to keep that around, if possible!), etc. Lather, rinse, repeat. I spent a lot of time in the Maintenance section of the OpenX admin system, trying to get the plugins to stop reporting errors.

Here’s a key problem: statistics in OpenX require the openXDeliveryLog plugin (possibly among others). Although it comes with the installer, it is not installed by default. There are empty folders under plugins where the plugins should be. Instead, the plugins are all zipped, inside the etc/plugins directory. Fortunately, when running OpenX as the administrator account, you can go to the Plugins tab and install them. Unfortunately, they still probably won’t stop reporting errors, because not all of the files end up in the right places. I had to extract openXDeliveryLog.zip and manually place the files for the deliveryDataPrepare plugin, a prerequisite for openXDeliveryLog.

A few more lather-rinse-repeat cycles and I got the plugins to stop reporting errors. But I still wasn’t getting stats. That’s when I discovered Frederik Schøning’s blog post, and was reminded that last time this happened, the problem was a second administrator account in the database. As he suggests, last time I was convinced this happened when the system got hacked. This time, I’m pretty sure there was no hack; the duplicate administrator account seems to have been created while I was running the installation/upgrade process (as hinted at above).

If this is the case, then the OpenX installer is broken on upgrade, and it creates a situation that breaks statistics reporting. Fortunately, it’s pretty easy, if you have direct database access, to remove the duplicate administrator account. Check the ox_accounts table for more than one record with account_type set as ADMIN:

SELECT * FROM `ox_accounts` WHERE `account_type` = 'ADMIN';

One of these should have account_id equal to 1 since it’s the first account you create. Delete the other:

DELETE FROM `ox_accounts` WHERE `account_type` = 'ADMIN' AND `account_id` != 1;

(Be sure to change 1 in that query to whatever the lower numbered account ID is, if it’s not 1. Or you could make the last bit = 80 or whatever the higher numbered account ID is. In my case, it was 80, but I suspect that will vary.)

Next, you need to make the proper admin account the system’s configured admin account again. That’s in the ox_application_variable table. As per Frederik Schøning:

UPDATE ox_application_variable SET `value` = 1 WHERE `name` = 'admin_account_id';

(Again, if your correct admin account’s ID is not 1, change the 1 to the appropriate value.)

As soon as I made these changes in the database, statistics were instantly working again. And I could stop thinking about OpenX… until the next crisis hits.

But before I stop thinking about it entirely, I want to pose a couple of questions to ponder:

1. Why, when updating an existing installation, does OpenX create a new administrator account?

2. Why are broken statistics the only (readily apparent) problem resulting from the presence of that second administrator account?

What could Apple buy with $100 billion?

In less than an hour, Apple is holding a conference call with the press to announce plans for its vast stockpile of cash, currently approaching $100 billion. What could they do with all of that money?

The most likely (and boring) suggestion is that they’ll start paying dividends to their shareholders. Bolder predictions include buying one of the cellular carriers (can you imagine Apple owning AT&T?) or possibly even Google. (OK, I don’t think I’ve actually seen that one, but I think they could do it.) Or, if they were Dr. Evil, they could hold the entire world ransom.

But let’s have a little fun, shall we? Here are some other things they could buy with $100 billion:

  • A large supreme pizza for every person on earth
  • A new stadium or arena for every team in the NFL, MLB, NBA and NHL
  • The entire GDP of Morocco
  • 500,000,000 iPod touches (“iPods touch”?)
  • Over 41,000 of the most expensive car on earth
  • 6,200,000 of the most popular car

Or, they could just give every person in America $320.93. Or pay off 0.64% of the national debt.

How not to do the Internet: a case study in bad web UX

My Subaru dealership’s website(s)

My Subaru Outback is due for service. Time to schedule an appointment. This is the kind of thing I just love to do online. No human interaction necessary!

Unfortunately, my dealership (who shall remain nameless, although I am happy to tell you that their shoddy website experience, which I am about to detail for you, was delivered by auto dealership web development behemoth Dealer.com) has not made the online scheduling option easy. And by “not easy,” I mean “practically impossible.”

First of all, for no good reason, the scheduling interface is built 100% in Flash. I’m pretty sure this scheduling system predates the iPhone, and Flash was at the time a de facto standard, but I’ve been building web applications since 1999 and there was never a time when it made sense to build a system like this in Flash. I deliberately don’t have Flash installed, except the version built into Chrome, so I couldn’t even load it in my browser of choice (Safari). Strike one.

I don’t think so.

Switching over to Chrome, I was able to go through most of the appointment scheduling process, picking the service I needed done, etc., until I got to the point of actually scheduling it, where I was stuck in an endless loop of picking days that were marked “available” on a little calendar overlay, only to get an error message stating “Not available due to shop capacity.” Over and over.

All but giving up, I spotted a curious item at the bottom of the page:

Ooh, this looks promising! Except… wait. How can clicking a link on my computer download an app to my smartphone? Maybe I should visit this site on my iPhone and then tap the link. Except… wait again. This part of the interface is in Flash, so I wouldn’t even see the link on my iPhone. So, what happens if I click it, anyway?

Are you kidding me? We are now at a place where I am interacting with a Flash-based web app on my computer, having it send to my phone an SMS message containing a link to the dealership’s mobile website. I’d even prefer a QR code over this convoluted mess. Strike two.

This is not how you do the Internet.

So I just decided to go over to my iPhone and try loading their website directly. It “smartly” (using the term loosely) redirected automatically to a separate iPhone-optimized version of the site, which I can only assume must be the mobile site the SMS would have linked to. (That is a reasonable assumption, right? We shall see.)

Here’s what I found when I navigated through the iPhone-optimized site to the service department’s page:

That “Service” button calls the service department. The “Contact Us” button is a link to a general-purpose email form. There are no other options.

“The mobile site will let you create, modify, and cancel appointments.” That’s what the SMS form promised. I decided (solely for the purpose of this blog post at this point) to fill out that form, but have the link emailed to me rather than sent as an SMS message. Not surprisingly, the email ended up in my junk folder. Also not surprisingly, the link goes to a different site than the version my iPhone was automatically redirected to when I tried to load the dealership’s main website. (So this is three separate websites they have now.)

I loaded this other mobile site on my iPhone, and it works!

Except, no, wait… it doesn’t:

Except, no, wait again… despite that weird error message, it actually does work:

(How many “except waits” is that now?)

But since it apparently only lets you edit existing appointments, not make new ones, I guess it really doesn’t work, after all. Also, don’t ask me why the iPhone version of the site doesn’t even have a link over to this version, but I guess ultimately it doesn’t matter anyway. Strike three. And four, five and six. In fact, let’s just call the game. And I’ll call the dealership.

Conclusions

What have we learned here? Aside from the fact that I am willing to spend far more time writing a blog rant about this problem than I am just calling to schedule an appointment over the phone, I hope I have impressed the value of web standards. If this site’s interface had been built in simple HTML and JavaScript, instead of Flash, which I can tell you is 100% possible — there is absolutely nothing in the interface functionally, and not even really anything in terms of the design or interaction, that could not have been done with HTML, CSS and JavaScript, even several years ago — then it would have worked fine in Safari without Flash, and it would work on my iPhone.

There has always been a lot of temptation to use Flash to do things that couldn’t be done with web standards, especially when the standards weren’t up to Flash’s capabilities. But succumbing to that temptation led to two trends that were profoundly harmful to the open web, and the effects of which we are — and I am literally — still experiencing today.

Bad UX as standard practice

First, since the sky is the limit when designing Flash interfaces, it created a feedback loop where designers took excessive liberties in experimenting with how basic interface elements like scrollbars and buttons should look and function. Then, project stakeholders who didn’t have the requisite background to make informed UX design decisions made demands that further pushed how these elements look and function. And since Flash had so few constraints on interface design, designers could accommodate those expectations, and the situation spiraled out of control.

This problem is not exclusive to Flash, of course. These days CSS and JavaScript offer tremendous potential for messing with standard UX conventions, and I’ve gotten myself into trouble with this on occasion. Customized scrollbars, fixed-position content, etc. Always remember, just because you can do something doesn’t mean you should. Things in the future will change in ways you can’t anticipate. Those changes are likely to accommodate existing standards, but not your customizations, no matter how brilliant they may be.

Flash as an industry

Second, as Flash grew in popularity it created an entire industry of Flash developers. Suddenly you had people whose livelihood depended upon building interfaces in Flash, regardless of whether or not Flash was really the right tool for the job. And as someone who has always favored open web standards, my perspective is that Flash is only the right tool for the job when it’s the only tool for the job. Even if standards couldn’t do something in quite as… ah-hem… “flashy” a way, if they could do it at all they should be chosen over Flash. I recognize this is a bit subjective, but one thing I can say about it with objectivity is that favoring web standards over Flash for a system like the one I’m describing here would almost certainly have obviated all of the problems I encountered this morning.

A metaphorical lesson to all technology workers: when your only tool is a hammer, every problem looks like a nail. Diversify your toolbox. It will serve your clients better by ensuring that you’re using the best tool for the job, not just the tool you know best. And it will serve your career better by not limiting your job options.

Future-proofing the web

There’s an even bigger reason now that having built this kind of system then using web standards instead of Flash would’ve been a good idea. If this site’s forms had been built with semantic HTML, it would be a comparatively trivial task to build new CSS with responsive web design techniques, allowing a single site to deliver the complete user experience on any device, rather than having to pay for and build three separate, wholly inconsistent, and all inadequate websites. By supporting and adhering to standards in the past, we could (and did) build websites that still work fine today. And by continuing to adhere to standards today, we can build websites that will still work fine tomorrow, even if we don’t really know what tomorrow’s devices will look like, or how we will interact with them.

My true path in life, finally diagrammed

It’s my birthday. What better time to reflect upon where I’ve been, and what I’ve become? Thankfully, Curiosity Counts (formerly a Maria Popova joint) has today linked to a flowchart, courtesy of Fast Company, that may explain it all.

A few of the details miss the mark: I taught myself BASIC, not Pascal (although I did try to make sense of Pascal while tinkering with my uncle’s IBM 5150 when I was 8), and I specialize in PHP, not Perl (distant cousins). But the lower left corner pretty much covers that, and also explains away the fact that I am married, for good measure.

Going back earlier in time, it even captures (and I’m being quite serious here) what might be the ultimate pivot point in my life: being massively obsessed with Atari but not getting into the next-generation game systems that followed it. It wasn’t for lack of interest; my parents simply refused to buy me an NES, and got me a Tandy computer (not a TRS-80, but close enough) instead. So thanks, Mom and Dad, for making a decision 25 years ago that set me on the path of lucrative uses of computers, instead of fantasy baseball and MMORPGs.

Here’s my version of the flowchart, with my path highlighted in yellow. Click the image below to see the full version, and be sure to check out the source, Taschen’s massive tome of infographics, while your at it.