New adventures in hi-fi… er, iTunes Match

As successful as iTunes has been in transforming both the music industry and the music listening experience, it has, from the beginning, been hamstrung by restrictions imposed by the outmoded, fearful major record labels.

Little by little, Apple has whittled away at those restrictions while managing to create a hugely successful business — iTunes has for several years been the largest music retailer in the world. First there was iTunes Plus: a boost in quality and a victory for users with the elimination of DRM copy restrictions. And now we have the real game changer: iTunes Match.

For $25 per year, you can now store your music “in the cloud.” iCloud, to be specific. That annual subscription allows you to create a centralized, comprehensive library of all of your digital music on Apple’s servers, and accessible from any of your computers and iOS devices. No more worrying about limited disk space or struggling with syncing issues. It just works.

In principle.

In practice? Well, I put iTunes Match to the test today. My music library poses a few unique challenges to this new service:

  • My library consists of over 18,000 songs, and more than 140 GB of data.
  • My main computer is a MacBook Air with a 128 GB hard drive, so I keep a “master” library on an external hard drive and a day-to-day library on the internal hard drive.
  • My main iOS device is a 32 GB iPhone 4, which has been syncing with the “day-to-day” iTunes library on my Mac’s internal hard drive.
  • SLP and I have our own iTunes accounts but have long desired to have a single shared music library.
  • My music library consists mostly of non-iTunes Plus tracks: a mix of DRM-laden 128 kbps iTunes tracks, ripped CDs, and tons of MP3s downloaded from Amazon.com.

With these factors in play, I had some specific goals for iTunes Match, roughly in this order:

  1. Move my “master” library from an external hard drive that sits on my desk, into iCloud where all of our devices can access it.
  2. Free up precious storage space on my MacBook Air and iPhone.
  3. Upgrade old DRMed 128 kbps iTunes tracks to higher-quality, DRM-free, 256 kbps versions.
  4. Consolidate SLP’s purchased iTunes music (around 600 songs) with my main library.
  5. Clean up duplicate tracks.

Spoiler alert: I pretty much knew going into it that the last of those items was going to get worse before it got better. But there were still plenty of surprises (good, bad and ugly) along the way.

The journey of 1,000 miles (or 18,000 songs) begins with a single step

I began my iTunes Match journey about two weeks ago, as soon as iTunes Match became available to the public. (For what it’s worth, I’m registered as an iOS developer, so I had access to the beta, but was never able to get it working properly.) The first goal was to get all of my music loaded into the system, and for the most part that went fine. Which is to say, it went… and went… and went… and w…e…n…t… fine. Loading the 4,000 or so songs I kept on my internal hard drive was fairly inconsequential. The process completed in a couple of hours while I went about my work that day. But then when the time came to fire up the external drive and load the remaining 14,000 or so songs… hmm. How can I put this? I guess the plus side was that I could leave it unattended and sleep, because it took three nights (overnight) to finish.

At that point I left things alone for a while, as I was too busy at the time to devote an entire day to organizing and cleaning up my music library. I did, however, get to play around with the overall iTunes Match experience for a week or so, and I discovered the following:

The good:

  • Having my complete music library at my fingertips on any device is amazing.
  • Streaming works great on the Mac, iPhone and Apple TV. Just pick a song and within a few seconds it starts playing.

The bad:

  • Browsing can be slow, sometimes painfully so, with a large library. This is especially a problem on the iPhone.
  • Cover art is often missing. I haven’t yet determined if it’s just not being downloaded, or if it’s not attached to the albums in iCloud, either.

The ugly:

  • Syncing an iOS device with iTunes on your computer can become a real mess. It’s hard to delete anything: like Michael Myers, no matter how many times you shoot him or stab him or stick a hanger in his eye, he just keeps getting up and coming back to get you. OK, bad analogy. But it almost feels that way.
  • Sometimes you don’t really want to remember just how many songs by Edison Lighthouse, England Dan and John Ford Coley, or Peppermint Trolley Company you own. It would be nice to have more filtering options than: a) just what’s on your device, or b) the whole shebang.

The big day arrives

Today I finally decided that I could afford to put off almost all of my real work for an entire day and devote my attention singularly to the task of getting iTunes Match fully synced, and SLP’s music fully integrated into the main library. To be honest, however, it’s not just today. I began the process at around 8:00 last night, worked until just after midnight, resumed from 7:00 to 8:30 this morning, then worked on it straight from 10:30 AM to 3:30 PM and again from about 7 PM to 10 PM, when I began writing this post. That’s 13 1/2 hours total, or approximately 2.66 seconds per each of the 18,266 tracks in my library. YMMV, as they say, but I’d guess it’s reasonable, if you’re trying to budget some time, to assume that you’ll need about 3 seconds times the number of tracks in your library. (And I’m still not really done.)

I took some notes today as I was going about things. Here are some pertinent observations:

Some things were just plain gone. I’m pretty sure this was the fault of my own carelessness in keeping my various pre-iTunes Match libraries in sync, but it’s worth noting that two conspicuous omissions in my library were The King Is Dead by The Decemberists and The King of Limbs by Radiohead. Coincidence?! I think… well, actually, yes, I do think it was probably just a coincidence. Luckily I was able to track down backups of both of those albums, but now I wonder what else is missing that I’m forgetting about.

“Matched” tracks are hit-and-miss. I’m sure Apple is relying on some very powerful algorithms to analyze each track in your library, in order to determine whether or not it matches a track that already exists on iTunes. It’s clearly not just relying on title-and-artist matching like the longstanding (and semi-useless) “Display Duplicates” option. One of the big selling points of iTunes Match is that if your music is available on iTunes, even if you didn’t buy it there, you’ll get the (usually higher-quality) iTunes version instead of the original version in your library, saving you time and saving Apple server space, as well. (Macworld’s Jason Snell has written an excellent tutorial on how to upgrade your tracks.)

The algorithms aren’t perfect, however, and I was annoyed to discover numerous cases where all but one or two tracks of an album were “matched” and could be replaced with 256 kbps iTunes versions, but the other tracks were rejected, for reasons unknown, and were stuck with the inferior quality versions I had to begin with.

Duplicates are a mess. Apple has done a lot to try to make it easy for you to find and weed out duplicate tracks, but you still have to do it. I appreciate that they don’t just assume which tracks you will or won’t want and automatically delete things capriciously, but I still wish there were a more efficient way to trim the excess.

Cloud symbols and error messages could use some clarification. Neven Mrgan has a great summary of the icons and his interpretations of their meanings, but I encountered too many dialog boxes today with useless statements like “This item is not eligible for iCloud” or “The track could not be downloaded because an unknown error occurred.”

If you’re trying to consolidate tracks from two separate iTunes accounts into a single library, you’re on your own. While the 10-device limit on DRMed iTunes tracks, and iTunes’ ability to be authorized for multiple accounts on a single device, allows for this kind of consolidation, Apple has not gone out of its way to support such activities. In my situation, I was dealing with a large number of SLP’s iTunes purchases that were no longer on any of our devices. I happened to have a spare Mac in my office with an empty iTunes library, so I logged into SLP’s iTunes account on that Mac and used the “Purchased” link in the iTunes Store to re-download all of her music in prep for eventual syncing with the main library.

But it wasn’t that easy. At first, a bunch of the songs wouldn’t download. I realized it was because they were still DRM versions, and that I needed to pay another $25 for an iTunes Match subscription on SLP’s iTunes account to get them. Even then, there were a number of weird issues with tracks being unavailable. Strangely, it seemed that in some cases, if I already had some of those tracks in the master iTunes library, and had already downloaded 256 kbps versions of them, it would not allow me to download them on this second computer. This leads me to believe that there is some hidden mechanism whereby Apple does still keep track of even the DRM-free tracks that have been downloaded, and if they’ve been “transferred” (as it were) to another user’s library, they become unavailable to the original user. This is just a guess, but it seems to fit my experience. (On a related side note, since this second computer was not yet authorized with SLP’s iTunes account, I needed to authorize it — which was triggered by attempting to play a song — before iTunes Match would work properly.)

What if your music is no longer available in the iTunes store? I’m sure this is one of the most commonly asked questions about iTunes Match, and I’m sure Apple has given very reassuring scripted answers, but it still remains as perhaps the biggest risk you take in trusting your music to the cloud. Tracks that iTunes fails to match and has to upload should be no problem, but once you’re relying solely on a “matched” track — or, for that matter, a “purchased” or “protected” track — you’re at the mercy of Apple and the record labels keeping the music available. I initially noted this as merely a point to ponder, but during the process of integrating libraries I encountered the problem firsthand. SLP had an album that was DRMed 128 kbps, but which is no longer available in the iTunes Store, at all. Luckily I had it copied to my master library already, or I wouldn’t have even known it existed. As it was, I was stuck with an album of low audio quality and that iTunes refused to load into iTunes Match. (It was “ineligible.”)

I hit upon a hokey workaround solution, one that is flawed mainly in that it results in further compression/degradation of the sound quality of the tracks, but at least it’s a way to get the music into iTunes Match. I burned a CD of the album, then re-ripped that CD back into iTunes, DRM free. (That’s the old school way of circumventing iTunes DRM, circa 2004.) It worked, but of course I’ll always know that the sound quality is sub-128 kbps. (Not that it matters much to me, as it’s an album I’ll probably never listen to.) This led me to a related discovery…

Burning a CD of DRMed tracks, re-ripping it, and uploading the results to iTunes Match will not get you “matched” DRM-free 256 kbps versions. Granted, my sample size here is pretty small — two tracks — but I suspect this is deliberate (if it’s possible). In addition to the aforementioned unavailable album, I found two other tracks from SLP’s library that stubbornly refused to load into iTunes Match, even though the rest of the tracks from the albums they were on were recognized and “matched” with no problem. So I burned them onto a CD, re-ripped the CD, and loaded the tracks into iTunes Match. No match. Just the further-compressed versions based on the original DRMed 128 kbps tracks.

Corrupted files? Are you kidding me? I had been wondering what might happen if files got corrupted, either during upload or download. Unfortunately, I found out. Just another meaningless error message with no real indication of a solution. I’m a few thousand tracks into the “upgrade” process so far, and to this point I’ve had four songs fail to download due to an “unknown error.” The behavior is the same in most cases: the song appears to download several times in quick succession. As soon as the progress bar gets to the end, it starts over again. After maybe 5 attempts, it stops with an error number (sometimes err = -100000, sometimes err = 11111). I think it may be necessary to contact iTunes customer support to resolve the issue, but I want to wait until I’ve finished downloading all of my music, to see if it happens with any other songs first.

So, is it all worth it?

I still have a nagging fear that some kind of catastrophic data loss is just around the corner, but so far I am inclined to say that iTunes Match definitely is worth it. It was delayed by a few weeks and still seems like it may have been rushed out the door, but I am hopeful that most of the current glitches and usability issues will be resolved over time. It would be nice if it “just worked,” as we Apple fanbois are so frequently inclined to say, but knowing the complexity of the task at hand, it’s a nearly superhuman achievement, even flawed as it is today.

At the moment I still have almost 3000 low-quality tracks that are eligible for an upgrade (using Jason Snell’s smart playlists), not to mention countless duplicates to weed out and a few other stray errors (in my nightmares, clouds have exclamation points) to contend with. But I think the biggest testament to the magnitude of Apple’s accomplishment is that it’s actually gotten me excited about “the cloud,” something I’ve looked upon disdainfully for years.

Top 5 Albums of 2011: The Contenders

It’s become an annual Underdog of Perfection tradition (even more of a tradition than calling this blog Underdog of Perfection). The annual “top 5 albums” post. Another tradition is this preliminary teaser post, wherein I announce “the contenders”: the albums I’ve purchased during the year that are under consideration for the top 5 list.

2011 has been an odd year for me, musically. I have been more prolific in my own musical endeavors than, perhaps, any year in the past — including three albums available on iTunes: my RPM project about Minneapolis, Scenes from the Busy Northern Metropolis, my running-inspired 40-minute electronic epic The Long Run, and the remastered/remixed compilation album Room 34 (Nokomis Beach). At the same time, I have found that many of my long-time favorite artists released disappointing, or worse, irrelevant albums this year, including but not limited to: The Beastie Boys, Cake, Coldplay, The Decemberists, Dream Theater, Fujiya & Miyagi, Radiohead, Wilco and Yes.

The year’s new music wasn’t all regrettable and forgettable, however, with bright spots from Adele, Foo Fighters and Halloween, Alaska, among others. But which albums will make the top 5? Check back in December to find out!

New Room 34 CD coming soon…

The time has come once again for me to engage in the extreme narcissism of producing a compilation album. I’ve recorded a crap-ton of music this year (even outpacing 2008, my most prolific year to date). Most of it I’m pretty proud of. Some, not so much. But this CD is just the good stuff. 13 tracks, 10 recorded in 2011 and 3 recorded in 2010. All remixed and remastered, with a kick in the pants in the form of boosted bass. So if you like your music with more low end than I’m accustomed to giving, this CD should suit you fine.

I’m putting the finishing touches on the masters this week, and am hoping to have the CD ready for production next week. In the meantime, here’s the cover art…

Update: So, yeah… this is available now. You can get it on iTunes or Amazon MP3 or, if you prefer physical media, direct from Kunaki. No free downloads yet. TBD on that.

How are Beatles albums like mobile-first web design?

When I’m meeting with clients and collaborators to discuss building websites, I like to make analogies. As the representative “tech geek” in most of these meetings, I find them the best way to convey the meaning of esoteric technical concepts, even if they’re sometimes rather strained. (I make car analogies a lot, for some reason.)

The other day I was explaining my two favorite (and overlapping) current trends in web design: Responsive Web Design and Mobile First. (How convenient that A Book Apart has books on both topics. I mention A Book Apart a lot in meetings, too.)

Suddenly in the meeting it occurred to me that mobile-first web design has an analogy with the production of most of the Beatles’ albums. In the early and mid-1960s, popular music was released primarily in mono format. Most of the market for these albums was listening to them on small mono turntables, not expensive stereo equipment. (And apparently at the time mono equipment could not properly play back stereo records.)

When the time would come for the Beatles to prepare the final mixes of their albums, the band members would join George Martin in the studio and carefully perfect the mono mixes. Then the boys would all head to the pub and leave George Martin alone to hastily assemble the stereo mixes as an afterthought. (And, frankly, it shows.) But somewhere along the way (in 1968, specifically) stereo had caught on enough that the last few albums (Yellow Submarine, Abbey Road and Let It Be) were mixed in stereo first.

The web is kind of like those Beatles albums. Up until now, websites were designed for “mono”: a computer screen. Eventually enough people started using the web “in stereo” (on mobile devices) that mobile versions of websites became necessary, but they were a hasty afterthought. But we are presently arriving at a time when a lot of people are doing a lot, if not most, of their web browsing on mobile devices, of a variety of shapes and sizes and capabilities. It’s starting to make sense not just to consider mobile versions, but to start with the mobile design.

Fortunately, we’re also at the point (to, as usual, strain the analogy) where the mono equipment can play back stereo records. There’s no need to design two separate websites, one for mobile and one for desktop. Responsive web design (via the magic of CSS3 media queries) lets us build one site that works on any screen.

But why mobile first? I see two main benefits, stemming from one main factor: the small screen size. By targeting the smallest screens first, you 1) focus on what’s most important, and 2) can more easily see what’s not important… or, at least, less important.

Mobile first fits well with the model of best-practices web design I like to promote. The decisions you make to create the best mobile experience will generally create the best experience, period. Granted, the needs of a visitor on their way to your office and checking your site on their smartphone may well differ from those of the visitor casually browsing your site on their iPad in bed, or from the customer placing an order from their office PC, but it’s easy enough to enhance the experience for users of those larger displays after the core needs of the mobile user have been addressed.

It’s an exciting time to be a web designer. Things are really starting to… come together.

(Come on, you knew I had to end with a Beatles reference.)

The Long Run

As I’ve been posting frequently on Twitter and occasionally here, on June 1 I started running. I’ve never been athletic or even particularly physically active, but at 37 I was beginning to feel the effects of my sedentary lifestyle. Thanks to the Couch-to-5K running plan and the fantastic C25K iPhone app, I’ve been able to finally get off my lazy ass and do something about it.

I had to take a break at one point due to shin problems and I ended up repeating a couple of weeks, but in the end I stuck with it and this week I finally finished the program. I hate to use the tired cliché “If I can do it, anyone can,” but it seems apropos. Couch-to-5K is really amazing, especially when an app makes it so easy to do. Now I’m regularly running 30-plus minutes every other day, and I’m looking forward to participating in my first 5K race next month.

The biggest challenge for me with the running lately has been finding interesting music to listen to while running. Combine that with the “itch” to record some new music, and the answer was obvious: I wanted to record an extended piece of music suitable for listening to while running. Over the past week I’ve worked on it in every spare moment, and now it’s finished. So, here I present “The Long Run,” clocking in at 40:37. Later in the week I’ll be posting it on my music site, both as an uninterrupted track and divided into 11 separate (but “gapless”) tracks. I’ll also be releasing it on CD in the 11-track form. But in the meantime you can hear (and download) it all right now on Alonetone:

alonetone.com/room34/playlists/the-long-run

For most of my longer runs I’ve chosen a route around Lake Nokomis, which I’ve commemorated with the album’s cover art, a collage of photos taken at and around the main beach this afternoon. (Yeah, I work fast with this stuff.)

Update: The album is now posted on my music site as well, and the CD is available for purchase. Full MP3 downloads (including the fabled, long-awaited single-track version) coming soon.