Tag: iCloud

Further adventures with the iCloud Photo Library… why can’t we just do this all on the web?

As I’ve mentioned previously, I have a problem with my iCloud Photo Library. A few problems, in fact:

I only add photos/videos to the library from my iPhone. I might occasionally take photos with my iPad, but I almost never need to add those to my library.
The way Apple “manages” storage for the iCloud Photo Library is woefully inadequate. As I noted in the post linked above, there are only three options: put everything on your device, let your device decide on some arbitrary amount of free space to keep, or don’t have the photos on your device at all.
The way the options work is not adequate for me with an unwieldy library of over 50,000 photos, and neither the storage space nor the need to have my whole photo library on my iPad and Mac.

I pretty much have no interest in the new Apple Intelligence features that are rolling out as we speak. But today I did learn there’s one intriguing new ability. (At least, I think it’s new.) The Photos app can now identify duplicates in your library, and purge them, retaining only the highest-quality version and merging all of the meta data.

I immediately took advantage of that feature, and removed about a thousand duplicate images.

But I still was confronting a very convoluted situation with my new iPad, which has only a meager 64 GB of storage. (I was seduced by a “Cyber Week” promotion to upgrade from my old 8th Gen to a 10th Gen that was on sale for $259, but I overlooked the fact that in so doing, I was going from 128 GB to 64 GB.) The whole damn library got loaded onto the new iPad before I could stop it, and now I’m dealing with trying to purge it.

There should be a simple and fast way to just delete the entire thing from a device, but there isn’t. Apparently turning off iCloud Photos on the device deletes everything, but… does it? It’s not instant, and there’s no progress indicator. The photos do seem to be very slowly disappearing from the device, but the lack of clarity is maddening.

Meanwhile, I also (finally) discovered something that I think should obviate all of this nonsense. You can very easily access your entire iCloud Photo Library from iCloud on the web!

Yes, just go to icloud.com/photos and log in to your account, and you have a fairly decent set of tools for interacting with your entire iCloud Photo Library without having to store any of it on-device.

One look at the crap I keep in my library should make it clear why I don’t need this filling up all of my devices’ storage.

Why isn’t this one of the default options? In fact, it seems like it would be a no-brainer for Apple to make the Photos app present an option to interact with the library in the cloud only. I know there are bandwidth and connectivity implications, so it would probably involve a reduced set of features.

I suppose, in a way, that’s what the option to optimize storage does — or at least what it’s intended to do — but I am still astonished that it works in such a way that even the reduced/optimized images can fill up your device storage. Why doesn’t it just keep the tiny thumbnails plus the meta data (and no videos!) on-device, and only retrieve larger versions when you interact with an individual image directly?

Again… maybe it actually is doing that, and it’s just that my library is so ludicrously large that it still causes a problem.

But I know I’m not the only person with a huge library (probably others are even much larger than mine), who only ever wants to add items to the library from their iPhone, not their iPad or Mac. It seems that in those extreme cases, they could provide a “web-only” option.

Or, at least… let people know you can shit-can your whole Photos library on the iPad and just access it through the web if you really need to get to your photos.

Sure, you lose cool things like the “memories” widgets on the iPad OS home screen if you don’t have the library on-device, but it’s a trade-off that is not only acceptable, but necessary in a situation like mine.

Sometimes, in their effort to make tools easier to understand by limiting how much they explain what’s going on behind the scenes, Apple actually makes their tools harder to use.

Is Apple ever going to properly address the inevitable, relentless growth of peoples Photos libraries?

I just got a new MacBook Pro with an M4 Pro chip, to replace my 3-year-old MacBook Pro with an M1 Pro chip. That older machine was fine, but, well… something in my gut just told me that if I was considering dropping two grand (minus some trade-in value) on a new computer manufactured in China, maybe I should do it before the end of 2024.

Anyway… I have the new MacBook Pro, and it’s pretty great. It’s essentially this year’s version of the same machine it’s replacing. The off-the-shelf model with a 14-inch display and a 512 GB SSD, at the same retail price. The upgrades are all in the Apple Silicon SoC: M4 Pro in place of the M1 Pro, with more cores and 50% more memory. A worthy upgrade to an already solid configuration.

I say “solid” because… well, I’d really like to have more disk space. I do a lot of video editing these days, and that requires a lot of storage. The problem is, Apple charges a ludicrous markup for increased disk capacity. And since external SSDs over Thunderbolt 5 are as fast as internal SSDs, I can get a 4 TB external disk for not much more than a meager 512 GB upgrade to Apple’s internal disk. The choice is obvious, even when weighing the convenience factor.

Which finally gets me (almost) to my point. Last night, after I had everything set up on the new Mac (I chose to manually install apps and copy files, so I could control what did and didn’t get moved over from the old Mac), I had over 300 GB of disk space free. This morning I sat down and was surprised to see I only had 180 GB free. What gives?

Well… what gives is the Photos app. I hadn’t even opened it — on purpose, for the exact reason I’m about to describe — but since I had logged into my iCloud account, the Mac “conveniently” automatically downloaded my entire Photos library from iCloud onto the Mac.

Do not want.

On my previous Mac, I had long since shuffled the Photos library off to an external disk. Honestly I never even really use Photos on my Mac, partly because my library is an unwieldy, 120 GB mess of close to 57,000 images¹. But mostly because Apple gives users (almost) no control over how the files are managed on their Macs:

So, basically, you have three options:

Fill up all of the space on your hard drive with your photo library.
Fill up all of the space on your hard drive with your photo library, until your Mac decides you’re running out of space, then let it decide how much space to free up.
Don’t have your photos on your Mac at all.

The default is #2, but it’s a pretty crappy option. Because for your Mac to know how much disk space to keep free, it needs to know how you intend to use it. If I’m not currently working on a video project, I don’t need much space. But if I am, then I can go from needing next to nothing to suddenly needing 200 GB or more of free space. Which means I always want to keep at least 200 GB free, in case I’m about to start on a video project. Once I’m rolling, I don’t want to be distracted with figuring out how to free up space.

Now here’s a twist: since, as I mentioned, external SSDs over Thunderbolt are essentially as fast as an internal SSD, you can easily edit a Final Cut project directly on an external disk. In fact, I do that a lot. But not always. I still want that free space.

Fortunately, Apple does provide a way to put your Photos library on an external disk. It’s easy and it works. But you shouldn’t have to do that.

The thing that really kills me is that it would be so simple for Apple to resolve this issue. All they’d need to do is add a slider to the settings tab from the screenshot above, letting the user set a minimum amount of disk space they want to keep free. Why doesn’t Apple do this? I feel like there’s a very intentional reason. And I think I’d be exasperated if I knew it.

¹ Apple is really to blame for the mushrooming library, too. Their automatic tagging by date and location, and AI-assisted tagging of subjects like individual people, pets, etc., make it so easy to find specific images that there is no incentive to put any effort into culling the junk from your library. Growing libraries lead you to pay for larger iCloud storage allocations to hold them. There’s a reason “Services” is Apple’s fastest growing revenue segment. But all of that aside — whatever, I really am not bothered by it — the one place Apple is completely shitting the bed on all of this is handling on-device storage.

The mysteries of Google Photos and what my videos are teaching its AI

For the past few years I’ve been shooting video, sometimes very long video, on my iPhone, and then transferring it to my Mac for editing.

I have two ways to get the videos to my Mac. I can use AirDrop for a direct device-to-device transfer. But the videos also automatically upload to iCloud in the background, so I can then open the Photos app on my Mac and find them there, and download them from “the cloud.”

Neither way is exceptionally fast, especially with large files, simply because they are large files, but they both have always worked seamlessly, and I can pretty much do them on-demand as soon as the videos are done recording.

Last night I recorded a performance of the big band I play in, using… gasp! …an Android phone. Specifically, a Google Pixel 3a. Yeah, it’s pretty old. But also new to me. I need an Android phone for testing my web development work, but since I don’t use it as my day-to-day phone, I didn’t want to spend a lot. I know the Pixel is a good phone, with a clean Android install, and the price was right ($150). Which is exactly why it’s the phone I chose to shoot the video with, because I was going to have to leave it unattended all night in a crowded auditorium, about 100 feet away from me. It was unlikely to be taken, but I didn’t want to risk it with my main iPhone or even a backup. (Plus, the Pixel actually does shoot pretty good video.)

Anyway… once I got home, I then had to contend with the problem I knew would be waiting for me. This is an Android phone, so there’s no iCloud. But I am also pretty heavily invested in the Google ecosystem, so I figured I’d have no issues transferring the videos to my Google Drive — or at least Google Photos — and get them that way.

There are issues.

First, I can’t find a way to just put them on my Google Drive, which is where I really wanted them. Yes, I can just jump over to photos.google.com instead of drive.google.com, without even having to log back in, but I operate frequently in Google Drive and I had never, before owning this phone, had any reason to even consider the existence of Google Photos. But, whatever.

Second, and I could be wrong about this, it seems like the files only transfer from the device to the cloud when you have the Photos app open on the phone. I have the phone running on my home wifi, which is connected to gigabit fiber, so the transfer should be about as fast as can be expected anywhere in the United States, but even though I left the app open for an extended period last night, it only uploaded one of the two videos. I gave up waiting on it and went to bed, and uploaded the second one this morning.

But here’s the real kicker: neither of the videos is “ready” yet in Google Photos, not even the one I uploaded last night. Maybe it hadn’t quite finished uploading then but still… what exactly is Google doing to get the videos “ready”? Why aren’t they “ready” the instant they’re uploaded?

I get that videos uploaded to YouTube need to go through some processing. YouTube does all kinds of crazy crap to videos. Some of it is transcoding and optimizing the files for streaming, which… yes. 100%. Please do that. Some of it is machine learning (a.k.a. “AI”) driven, analyzing the content to make smart chapters and such, as well as detecting content that is illegal or violates terms of use. I see the benefits. And some of it is for the benefit of corporate overlords, both at Alphabet and elsewhere, such as detecting copyrighted music. Well… honestly there are pros and cons to that. But it’s not the point of this post.

The point is, I get why that kind of processing is happening to YouTube videos. But none of it should be happening to private videos uploaded to your Google Photos account. I suppose I should give Google the benefit of the doubt and assume that it’s strictly doing the AI scans for illegal content. (I think you know the illegal content I am talking about so I am not going to spell it out.)

Apple’s recent efforts to implement technology to do on-device detection of that type of content recently got a lot of heat for its privacy implications — even though on-device scanning is far more private than in-the-cloud scanning. But Apple always takes the heat for things because it’s Apple, despite Google, Amazon and Facebook engaging in much more egregious versions of those things.

The thing about Google in particular is, they’re all about amassing TOTAL WORLD KNOWLEDGE. Not in an inherently nefarious way (although maybe there’s no way for that not to be inherently nefarious). But there are always nefarious implications in what they’re doing. So whatever is going on as I’m waiting for my videos to be “ready,” the one thing I know for sure is that Google’s AI is learning something from them.

Update: I realized after I posted this that maybe I should… uh… google it. Yes, Apple themselves have provided instructions on how to transfer photos and video from Android to Mac. I realized that I could maybe use this USB-C to USB-C cable I have sitting right here at my desk to make a file transfer!

iCloud Drive: Don’t do what I did!

Being deeply immersed in the Apple ecosystem, a couple of years ago I made a decision:

I’ll move all of my work files onto iCloud Drive!

I work (as in, write code and edit image files) mainly on my Mac. But I was seduced by the possibility of accessing all of my work files in a pinch on my iPad (which I still had at the time) or even my iPhone. Plus, since my files would be in “The Cloud”, I could even access them from another computer (or from my Mac when booted into Windows) if I needed to, by logging into my iCloud account from a web browser.

It seemed… so obvious. So perfect.

Umm… maybe not.

For the past two years, I have been constantly fighting with iCloud Drive. One of its signature features is that it can manage disk use on your Mac automatically, so as your hard drive fills up, it deletes files you haven’t used in a while, keeping them safely in the cloud while freeing up disk space on your Mac. And with my MacBook Pro sporting a (meager?) 256 GB hard drive, with 40-odd GB allocated for a Windows partition, and over 60 GB occupied by Logic Pro X sound samples, my drive is filling up constantly.

While this is great in principle, it is completely unworkable in practice for three interrelated reasons:

If you have a large amount of data in play here (for me, it’s in the vicinity of 100 GB), iCloud Drive may get to a point where it is constantly transferring data. If you’re not on a gigabit fiber connection, this can both use up all of your Internet bandwidth and take ages.
Because #1 is taking place constantly, if you do find yourself needing to grab one of those files that has been deleted locally (as indicated in the Finder by a cloud icon with a down-pointing arrow), you may find yourself waiting several minutes for the file to become available, even if it’s small (as in, under 1 MB).
In an effort to make this all appear seamless to the user, the Finder represents cloud-only files as… regular files. But they’re actually just pointers with a hidden .icloud filename extension… as you’ll find if you ever try to perform Finder actions inside another program, such as syncing files to a web server using Transmit.

All of this might be tolerable if Apple gave you any control whatsoever over which files get deleted locally. But they don’t.

It gets worse.

What’s worse than getting stuck in this situation? Trying to get out of it. It’s like quicksand… the more you struggle against it, the faster and deeper you sink into it.

I made the decision earlier this week to extricate myself from the iCloud Drive nightmare, by buying a 250 GB SanDisk external SSD. First off, a little unpaid plug for this drive… it is awesome. It’s super light and small, seemingly at least as fast as the internal SSD in my MacBook Pro (in that I am able to transfer multiple gigabytes of data in seconds), and it even looks cool. I’m going to be using it all the time, so I’m actually considering putting adhesive Velcro on both it and the top of my MacBook Pro so I can keep it permanently attached. (Which says a lot about how much my regard for Apple has fallen lately — in the past I would never have sullied the exterior of an Apple laptop with something adhesive.)

So anyway, external SSD acquired, my goal was to start transferring my files from iCloud Drive over to the SSD.

Uh… good luck with that.

Because I’m now at a point where I have more than double the amount of data stored on iCloud Drive as I have available space locally, a majority of my files are now only in “The Cloud.” Ugh. Which means waiting for all of that data transfer stuff to happen. If only I could, somehow, bypass this broken process, I thought.

There has to be a way.

tl;dr Nope.

So, here’s the thing. I’ve been using iCloud Drive for the bulk of my cloud-based file storage, but I do use other services as well. I have Google Drive. I have Dropbox. I know how they work.

Specifically, I know that you can, y’know, like, select a folder and download the entire thing as a zip file.

So I thought to myself, I’ll just go to iCloud in a web browser and do that! Download the whole friggin’ thing as a big zip file, or maybe a few zip files, and be done with it.

Nope.

For whatever reason, iCloud doesn’t let you do that. Probably because of the whole seamless “It just works” Kool-Aid drinking song everyone in Apple land has been singing. (Myself included, mostly.)

You can only download individual files, not folders, from the iCloud web interface. It does let you select multiple files at once, but only within one folder.

Check out this delightful thread full of know-it-all asshats whose response to a legitimate question — why doesn’t Apple allow this thing that every competing service does? — is to challenge the validity of the question and the intelligence of the questioner. (That thread is now closed so I’ve just opened my own new thread on the topic. Watch this space for trolls!)

There. Are. Plenty. Of. Reasons. A. Person. Might. Have. For. Needing. To. Do. Something. That. You. Have. Not. Previously. Considered. Stop challenging their premises and answer their question, or shut the hell up.

Whew.

I ended up “solving” the problem by resigning myself to the fact that it wouldn’t be completely solved. So instead I took the drastic approach of temporarily logging out of iCloud completely, just so I could strand the files I did have saved locally, and copied them to the SSD.

Then I logged back into iCloud Drive and tried to get it to stop syncing my files by unchecking the Desktop and Documents Folders option.

The only problem is, I didn’t have my work files in those folders. I had them in a separate top-level folder in the iCloud Drive that I created myself. Because, you know, you can do that and it didn’t seem like a crazy idea or anything.

It was.

I discovered this morning that even though I had done all of this and tried to purge the nightmare of constant iCloud Drive syncing from my Mac life, once I had logged back into iCloud, the Mac went right back to quietly, constantly, syncing that iCloud Drive data on my Mac. As I type this, I have a Finder window open to my iCloud Drive and in the status bar it says “downloading 120,079 items (36.14 GB of 48.66 GB)”. Fun!

So, my new plan for today is to watch that window, and as the little cloud icons next to individual folders goes away, I’m copying those folders to my SSD and then deleting them from iCloud. My assumption is that as I do this, I am freeing up more local space and iCloud will continue to download the remaining items, and eventually I’ll have everything transferred over.

But please… do yourself a favor and don’t do what I did. iCloud Drive is not suitable for professional use.

How to get iWork (Pages, Numbers, Keynote) apps to stop defaulting to iCloud when saving

Ever since I semi-fully embraced iCloud, I’ve found that the iWork apps — Pages, Numbers and Keynote — always default to wanting to save every new document in iCloud, which I never — well, OK, almost never — want to do. It’s fine that it’s an option, but I want the default to be saving to my local hard drive (which, actually, means saving to my Dropbox account).

It didn’t take much effort to find this thread on Apple’s support forums, but the first suggested solution — turning off “Documents and Data” in System Preferences → iCloud — seemed draconian. With this option you can never sync your documents to iCloud.

A little further down the thread I found the “real” solution, courtesy of “Bernie_uk”, which was important enough for me to want to share here.

It requires opening up Terminal, but it’s not too scary. You just have to run this command:

defaults write NSGlobalDomain NSDocumentSaveNewDocumentsToCloud -bool false

This command doesn’t turn off anything in iCloud; it just tells the system that your default should be saving files to disk, not to iCloud. Note that since this is a global setting, it will affect not just iWork, but any other apps that use iCloud’s “Documents and Data” syncing. (I guess.)