Mac App Store “sandboxing”: perspective of a long-time Mac “power user”

In the web world I reasonably qualify for the title of “developer” as I spend most of my work day writing server- or client-side code. In the iOS world I nominally qualify as a “developer” in that I have paid the $99 fee to join Apple’s iOS developer program, although I have yet to do anything with it besides download early betas of iOS 5 that I never even bothered to install because I don’t have an extra device to experiment with. But in the Mac world, I am nothing more than a “power user” at best. Aside from a couple of Automator scripts (which do not count), I’ve never created a Mac application and doubt I ever will.

Therefore I’ve kept a healthy distance from Mac app developers’ criticisms of the ongoing “iOS-ification” of the Mac, specifically where the Mac App Store is concerned. I don’t use most of the iOS-like features in Mac OS X Lion, and probably won’t in Mountain Lion either, but I do use the Mac App Store, occasionally. I think it works pretty well. It’s actually gotten me to buy some apps — games mostly (of course), plus a couple of utilities like Space Gremlin (yes, that’s a utility, not a game) — for the Mac that I probably would neither have known about nor bothered to pay money for otherwise.

But there are some things that, as a power user, I recognize as problems with the Mac App Store, specifically concerning the “sandboxing” requirement that will take effect on March 1. Essentially, in order to be carried in the Mac App Store, an app cannot access the Mac’s file system except in very limited ways dictated by the OS. It’s a huge step towards making the Mac behave like iOS, which is mainly beneficial in two ways: it improves security by reining in potentially malicious applications, and it simplifies the user experience for novice computer users in a tried-and-true way.

It may be reasonable to question at least one, if not both, of those benefits however. Regarding the latter, I suspect that most would-be Mac owners who have too much difficulty understanding the Mac interface, and who would be better served by making the Mac more iOS-like, would really be better served by not buying a Mac at all, and just getting an iPad instead. Since Apple sold more iOS devices last year than all of the Macs it has sold, ever, that seems like a no-brainer. For a large and growing majority of its customers, Apple is the company that makes iPhones and iPads, not the company that makes Macs.

But it’s the former issue — the security-enhancing measure known as “sandboxing” — that is more troublesome for established Mac users like me. A lot of Mac apps need unfettered access to the file system to do what they do. Especially for those of us “power users” who write code — for the web or otherwise — we need applications that not only have full access to the file system, but that can even let us see “invisible” files. (I always have Mac OS X set to show invisible files, since I frequently need to work with .htaccess files, for instance. Did you know that any filename that begins with a period is automatically hidden by the OS by default? It’s an old Unix thing, designed to help keep users from accidentally deleting critical system files.)

A real Mac app developer has weighed in on the problems posed by the sandboxing requirement for his company’s app, SourceTree, and how, as a result, Atlassian has had to remove SourceTree from the Mac App Store.

Fortunately, unlike with iOS, the Mac App Store is not the only way to get applications onto your Mac, and I think it’s a reasonable assumption (but by no means guaranteed) that the Mac will always allow power users to do things the “old fashioned way” and install directly applications that do not conform to the App Store’s strict requirements.

But a look at the features of the upcoming Mountain Lion version of OS X gives pause. Clearly Apple wants to move the Mac more and more in an iOS direction, and clearly even beyond the growing sibling relationship between the two platforms, just on the Mac itself, Apple is putting a lot of effort into streamlining and simplifying things, making things on the Mac work, as Grubes said yesterday, “closer to how things should be rather than simply how they always have been.”

But how things “should” be is subjective, especially when it comes to what “real” applications on a “real” computer can reasonbly be expected by “power users” to be “able” to “do.” (Quotation marks indicate uncertainty.)

There’s been a lot of talk over the past few years about how we’re entering into a “post-PC” era. With the proliferation of smartphones and tablets (90-some percent of the latter being iPads, of course), new interaction methods and increasingly mobile-first web experiences, what is to become of the trusty old PC (Mac or Windows)? What surprises me most in all of this is that Apple and Microsoft are leading the charge to turn traditional computers into post-PC devices. Both Mac OS X Mountain Lion and Windows 8 are radically diverging from their traditional interfaces into new directions inspired by their mobile siblings.

In some ways this is an exciting and fascinating time. For about two decades now, Mac OS and Windows have been very similar, presenting nearly identical ways of interacting with computers. But iOS and Windows Phone (which is coming to Windows 8 as the Metro interface) are very different from each other. The next standard UI has not yet been established. We haven’t seen this kind of competition and variety since the early 1980s, before Microsoft’s desktop OS dominance was established.

But it’s not the early 1980s. We’re not living in a time of unprecedented invention and discovery. We have nearly 30 years of GUI-based computing experience under our collective belts, and the conventions of standard GUI interfaces (as represented mainly by Windows and Mac OS X) have become expectations of the “power users” who rely on them not just as an alternative form of entertainment while sitting on the couch in the evening, but to do real work — creative, technical, business-oriented work.

That’s not going to change overnight as February passes into March. And it may not change significantly in our lifetimes. But if Apple aggressively restricts what applications in the Mac App Store can do, in ways that prevents users from getting work done, while simultaneously pushing the Mac App Store as the primary (and eventually only) way to install applications on a Mac, they will effectively kill the Mac as a tool of creative professionals, which is pretty much the only thing that kept Apple alive through the ’90s in the first place. Sure, Apple doesn’t really need us now (and it would still have us anyway… I’m not giving up my iPhone anytime soon), but making the Mac too much like iOS doesn’t necessarily make it better. It just makes it… unnecessary.

Test 1-2, test 1-2

There is no substantive content in this post, really. At least I suspect as I write it that there won’t be. I’m merely testing the capabilities of the latest version of the WordPress app for iOS, released today, which I disparaged in passing (based on my experience with a much earlier version) during a podcast while pondering the merits of Tumblr.

Already I see many improvements since the last time I bothered to try using this app. There’s actual formatting in the text entry box, for one. And you can even insert pictures taken with your phone!

20120216-212630.jpg

Yes. I just took a picture of what’s right in front of me, right now, which happens to be the view from my living room couch. And yes, we still have Christmas Valentine’s Day lights up. You got a problem with that?

Clearly, Tumblr has lit a fire under WordPress’s backside, and the iOS app is one way the Automattic bucket brigade is attempting to extinguish it. Time to give this app another chance!

Mountain Lion. Clever?

Apple just announced the next version of (Mac) OS X: Mountain Lion. And they did so in an rather unusual fashion. Grubes has the scoop:

The recurring theme: Apple is fighting against cruft — inconsistencies and oddities that have accumulated over the years, which made sense at one point but no longer — like managing to-dos in iCal (because CalDAV was being used to sync them to a server) or notes in Mail (because IMAP was the syncing back-end). The changes and additions in Mountain Lion are in a consistent vein: making things simpler and more obvious, closer to how things should be rather than simply how they always have been.

But a lot of the chatter (chirping?) on Twitter concerns the name:

I “get” what Apple’s doing with the name. Mountain Lion (10.8, presumably) is to Lion (10.7) as Snow Leopard (10.6) was to Leopard (10.5): a refinement, a continuation of the same direction in OS X’s evolution as the version that preceded it.

But it is rather odd, if you think about the actual cat names involved, especially since a mountain lion is essentially the same thing as a cougar, also known as a puma or… a panther, which Apple already used as the “big cat” codename for Mac OS X 10.3 way back in 2003.

Beyond that, Lion always seemed like something Apple was building up to… the “king of the jungle.” That version 10.7 was named Lion seemed to suggest the “big cat” lineage, version 10.x of Apple’s Mac OS, and perhaps even the “X” (which is pronounced “ten” after all) was done. And here we have… Mountain Lion? A decisive step backwards in the awesomeness of the big cats. Heck, mountain lions have even been spotted here in Minnesota for crying out loud!

So… anyway… I think the point is: Apple is not taking this whole “big cat” thing too seriously, and neither should we. Mountain Lion looks pretty great. I can’t wait to try it out!

Update: After some research, it turns out Apple already used Puma, too, for 10.1. But as I recall, they didn’t start using the big cat codenames in marketing until 10.2, Jaguar. And I still have the Jaguar mousepad on my desk to prove it!

Where’s the “tech” in tech blogging?

A minor scandal has gently rocked the tech world lately, as the CrunchFund-…uh…-funded startup Path was caught uploading its users’ entire Address Books from their iPhones to Path’s servers. Path quickly rationalized, then apologized, then did a reasonably good job of making amends, but it stirred up a lot of concern (which probably should have been stirred up much sooner) about social networking platforms harvesting too much private Address Book data, and Apple’s iOS, in particular, not doing enough (which is to say, anything) to warn users that this was going on.

But more attention is now being drawn to the playground fight between former TechCrunch writer, current CrunchFund investor, Michael Arrington acolyte and self-serving tech blogger M.G. Siegler, and former Fake Steve Jobs, ongoing Newsweek technology editor and self-serving tech blogger Dan Lyons. I’m quickly learning that while it’s not possible to agree with both of them, it’s quite easy to disagree with both. In this situation neither seems nearly as concerned with the actual technology involved — nor in finding a technical solution to the dilemma of startups that want to offer their users the benefits of social integration while respecting privacy — as in proving their own awesomeness while ripping the other to shreds.

In short, it’s becoming a lot harder to see where technology fits into all of this ostensible tech blogging.

For quite a while I’ve been on M.G. Siegler’s side. To be honest I haven’t paid Dan Lyons much attention. He seemed like a gimmicky hack when he was pretending to be Steve Jobs, and just a bitter hack afterwards. M.G. Siegler isn’t necessarily much better. I have long loathed Michael Arrington’s work and anything associated with it (as I’ve documented here before), with the lone exception of Apple-focused M.G. Siegler. Which isn’t to say I am a huge Siegler fan (despite how frequently I have cited him on The Undisciplined Room); his favorite topic, I have learned, is not Apple but himself. He also peppers his writing with far too much gratuitous profanity for my taste. (Not that I don’t swear like a fucking sailor most of the time, but swearing just to make your writing seem edgy has been passé since shortly after Rolling Stone started doing it 40 years ago.) But even if I bristle at his style, I usually find his writing engaging, if not informative.

When the late, great Clara Peller famously asked “Where’s the beef?” (yes, I just went there), she wasn’t calling for a grudge match between two Silicon Valley pseudo-journalists; she was looking for something of substance. And substance is in as short supply in money-driven tech blogging as it was in fast food in the ’80s. (Thank goodness for Wendy’s and its public service campaign that fixed that, once and for all.)

To find the substance, we can go back to a voice that has been engaged in this Path discussion from the very beginning, one of my new favorite (real) tech bloggers, Matt Gemmell. Not only is he focused on solving the problem of Address Book privacy rather than on inflating his own stature in the blogging world, he actually knows what he’s talking about and has described a brilliant solution to the problem that I think every company that is collecting personal data from its users should employ.

In short, it’s easy to figure out which tech bloggers to side with: the ones who actually talk about technology, instead of each other.

Uh… excluding this post… by me… of course.

Fortunately for me, I don’t have enough of an audience to matter. (And with that attitude I never will either, I say in my head, in the voice of Agnes Skinner.) The majority of my traffic is one-time visitors searching Google to figure out why their Macs aren’t working right, or trying to get Drupal out of their lives, or wondering why open water on maps scares them so much, or trying to make sense of that weird Brooks Brothers logo. You know, the important stuff.

Migrating primary keys in a CakePHP site’s database from GUIDs to integers

If you are a CakePHP developer, and find yourself in a situation where your database is using GUIDs but you want to switch to using integer-based primary keys instead, this post will describe how I dealt with that exact situation. (Actually, the post will do that even if you’re not a CakePHP developer; it just probably won’t be of any interest to you.)

Some unnecessary background details you can skip if you’re in a hurry

I may be the only CakePHP developer silly enough ever to have used GUIDs (those crazy pseudo-unique 36-character hexadecimal strings) instead of plain auto-increment integers as primary keys. Maybe not. In any case, it only took me one CakePHP project to realize the folly of the approach and I quickly abandoned GUIDs on my second CakePHP site.

Now the time has come, nearly four years later, to finally update that first CakePHP site to the latest version of cms34. There will surely be numerous challenges involved in the upgrade, given the extensive evolution the system has undergone in the intervening years. But without a doubt the biggest, or at least the first, of those challenges is getting the data converted from using GUIDs to integer-based primary keys.

The reason I used GUIDs in the first place was that I have a few multi-purpose cross-reference tables in the structure, and I wanted to just be able to refer to a single ID field as a foreign key, without having to keep track of both the table/model and an integer ID. Kind of silly, in retrospect, especially since I quickly realized how important it was to be able to easily identify which table the foreign key pointed to. At the time I was singularly focused on trying to keep the database as stripped-down basic as possible, to a self-defeating point.

But my early CakePHP learning curve is a bit beside the point here. The goal of this post is to document the process I am undertaking to migrate the site, for the benefit of anyone else who finds themselves in a similar situation. (Hopefully, for you, the idiot who used GUIDs was not yourself.)

Step 1: Preliminary set-up

First, a qualifier: the way I am doing this will result in a lot of gaps in the sequence of integers used in the individual tables after the migration. If the actual values of your integer primary keys matters to you, you may not want to take this approach. I can’t imagine a reason why the values would really matter though.

The idea in this step is, in essence, to convert each GUID in the system into a unique integer that can be used later in the update process. There’s no reason why the integers have to be unique across tables any longer; it’s just that having a definitive reference, in a single table, of how each GUID maps to an integer will make some of the later steps in the process easier. If you are compelled not to skip numbers in individual tables, you could add an extra field called table_id, and increment table-specific IDs in that field. But that would require extra code in the next step, and it just seems like an extra complication without any real benefits.

Do not do any of this on a live site. Set up a duplicate copy of your site somewhere else, and do all of this conversion work there. Also discourage users from updating the existing site while you’re doing the migration, if possible. This conversion can be set up as a repeatable process, however, so you could always do some test runs, and then once you have everything working reliably, get a fresh dump of the live database again, minimizing downtime.

In your copy database, create all of the tables for your old database and load in the data. Also set up empty tables for the new database (which hopefully doesn’t have overlapping table names; if it does, add a prefix to the table names for the old database). Once you have both the old and new tables in your database, and the old tables loaded with data, you’re ready to proceed.

Step 2: Build and populate a GUID map table

Assuming you’re using MySQL, this will take care of it:

CREATE TABLE IF NOT EXISTS `guid_map` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`table` varchar(255) NOT NULL,
`guid` char(36) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1;

If you have adequate permissions to create the table from within a PHP script, you may as well start building one, because you’ll be adding to it next anyway:

<?php
// Connect to database (replace values as appropriate)
$db_name = 'database'; // Set to your database name
$dbconn = mysql_connect('localhost','username','password',true);
$db = mysql_select_db($db_name);

// Build guid_map table
mysql_query("CREATE TABLE IF NOT EXISTS `guid_map` (
    `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
    `table` varchar(255) NOT NULL,
    `guid` char(36) NOT NULL,
    PRIMARY KEY (`id`)
    ) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1"
);

Next, you’ll want to build a PHP script that will crank through all of your old tables, grab all of the GUIDs, and write them into this new table. Here’s a snippet of PHP to get the job done. You’ll need to decide how you want to run it; it assumes a database connection is already established, so you could just drop it in as an action in one of your controllers or you could build a stand-alone PHP script and add the necessary database connection code at the top.

Note: This works with direct SQL queries, not CakePHP’s model methods. That’s just how I did it. Also, when I build PHP utilities like this, I run them in a browser and like to dump status information as basic HTML. If you prefer to run this at the command line, or don’t care about a verbose dump of the details, you could modify/remove any lines here that contain HTML.

// Clear current GUID map data (makes this process repeatable)
mysql_query('TRUNCATE TABLE guid_map');
if (mysql_error()) { echo '<p>' . mysql_error() . '</p>'; exit; }
echo '<p>GUID map cleared.</p>';

// Get tables
$old_prefix = 'old_'; // Set to prefix for old table names
$rs = mysql_query('SHOW TABLES');
$tables = array();
if (!empty($rs)) {
    while ($row = mysql_fetch_array($rs)) {
        if (strpos($row[0],$old_prefix) === 0 && $row[0] != 'guid_map') {
            $tables[] = $row[0];
        }
    }
}

// Loop through tables, retrieving all GUIDs
foreach ((array)$tables as $table) {
    $sql = 'SELECT id FROM `' . $table . '`';
    $rs = mysql_query($sql);
    
    // Loop through rows, adding to GUID map
    $error_count = 0;
    $mapped_count = 0;
    if (mysql_num_rows($rs) > 0) {
        echo '<hr /><h3>Processing ' . mysql_num_rows($rs) . ' rows in table: ' . $table . '</h3>';
        while ($row = mysql_fetch_assoc($rs)) {
            $sql = 'INSERT INTO `guid_map` (`table`, `guid`) VALUES (\'' . mysql_real_escape_string($table) . '\', \'' . mysql_real_escape_string($row['id']) . '\');';
            mysql_query($sql);
            if (mysql_error()) { echo '<p>' . mysql_error() . '</p>'; }
            else { $mapped_count++; }
        }
        echo '<p>Mapped ' . $mapped_count . ' rows with ' . $error_count . ' error(s).</p>';
    }
    else {
        echo '<p>No data rows found in table: ' . $table . '</p>';
    }
}

Once you’ve run this script, your guid_map table will now contain a row for every row of every table in your database. And you now have an integer ID you can use for each of them. And what’s extra cool about it is that it’s repeatable. Run it as many times as you want. Get a new dump of data from the old site and run it again. Fun!

The next step is where things get a lot more complicated, and a lot more customized to your specific data structures. I’ll be using one of my actual data tables as an example, but bear in mind that your system is unlikely to match this schema exactly. This is just a demonstration to work from in building your own script.