Permission Denied
I mentioned a while back that my 2012 iMac’s hard drive finally died, which led to some tricky choices: to replace it altogether, or replace the hard drive with an SSD, or with another hard drive (not recommended nowadays), or a fusion drive where the system treats a dual SSD and HD as a single drive, or—my eventual choice—a 480GB SSD and a 4TB hard drive installed as separate drives, with system and everyday files on the former and archival files on the latter. That way I got the full speed boost of running my system and apps on an SSD, with the space for archives of a honking great hard-disk. I got some extra memory installed while I was at it, so that the revamped Mac would last me at least a few more years and save having to spend two grand on a replacement.
Everything went fine, and I had a recent enough backup that I lost no files worth worrying about, although there were the usual software updates after the transition to macOS Mojave. But in copying stuff over from my backup, I made a couple of mistakes that led to some month-long headaches. This post serves as warning and solution, if you’re ever in the same boat.
Because I was copying from a backup of a single drive onto two separate drives, I did it in two stages, starting with copying my archival files onto the hard-drive to make sure they were safe. Once that was done, I deleted them from my backup (though I still had a backup of that backup, just in case) to reduce its size to half that of the SSD, so that I could import the remainder onto the SSD using Migration Assistant, bringing over all of my apps and user accounts. That worked well enough, I thought. Once everything was safely in place, I deleted the original admin user account installed by the repair place, leaving only my usual user account as admin.
And Here My Troubles Began.
I’d forgotten about permissions.
Because Mac OS X (um, macOS) is Unix-based, every file, folder or disk has a set of associated permissions, assigned to Owner, Groups, and Everyone, giving permission to read a file (or a folder’s contents), write to it, both, or neither. A user’s files are assigned permissions of Read, Write, and Execute in different combinations, to owner, groups and everyone else, where the owner is the user.
My archival files used to be stored in my user folder on my old set-up, but I’d now copied them onto what was effectively an external hard-disk (mounted internally, but separately from the boot drive). In doing so, the Mac had changed the ownership to the user doing the copying, which because of the order of the steps outlined above was the original admin user account installed by the repair place. Which I’d now deleted.
Suddenly, I found that anytime I wanted to do anything with one of these copied files on the archival drive, I would be prompted for an admin user password. And I mean anything: move it, copy it, rename it, delete it. Moving files around en masse while sorting out my new set-up was a serious pain. And even if I copied the file to the SSD, it retained these problems.
Doing a Get Info on such files showed that the permissions had become seriously snarled up, showing multiple owners with different levels of permission, and an owner that showed up as “Fetching...” but never resolved (presumably, this was the user I’d deleted).
Okay, so I had to repair permissions, which anyone who’s had to use Disk Utility on a Mac in the past will be familiar with. But wait: since the macOS-before-last, we haven’t been able to do that in Disk Utility, because Apple have now built it into the startup process instead. So persistent permissions issues of this kind aren’t so easily fixed.
This helpful change on Apple’s part seems to have caused such widespread problems that they’ve quietly announced a technical fix for permissions in users’ Home folders. I followed those steps, which did seem to fix the files in my Home folder on the SSD. But my archival files were on another disk, and unfixed—and unfixable—using this method.
It was getting to the point where I was contemplating booting my iMac as a target disk from a MacBook running an older version of Mac OS, so that I could run a copy of Disk Utility that still allowed me to repair permissions. But then I read somewhere that Disk Utility only ever repaired permissions on system files and applications, and wouldn’t touch the swathes of regular files I was concerned about.
Was I going to have to drop a hundred quid on DiskWarrior or some other third-party disk-repair app? Or reformat the internal HD and re-copy the files onto it using my regular user account?
Fortunately, no. The answer was to fire up the Terminal and use a few straightforward Unix commands to fix them all at once. Fixing the permissions on any particular file involves the following steps:
sudo chown username /path/to/file
This changes the owner of the file to your username (so you’d replace username with the desired owner here). Starting the command with sudo means that it’s carried out as root, the top-level superuser of Unix systems (you’ll be prompted for a password). Replace the /path/to bit with the Unix pathname of the file: on an external drive this would be something like
/Volumes/ExternalDriveName/"Folder Name"/"File Name"
(with the quotes for any names containing spaces). But you don’t have to know all of that if you’re using Terminal: just type a space after the username, then drag and drop the file onto the Terminal window, which will insert its full pathname at the end of the command, and then hit return.
Now you own the file, but so do all those other duplicate owners that have been messing things up. These, it turns out, are stored in something called an Access Control List, which records which additional users share ownership of a particular file. To clear this ACL and reset it to the main owner (which is now your username), use the command:
sudo chmod -N /path/to/file
The command chmod changes the mode of the file, and the option -N strips away the file’s ACL. You can use the drag-and-drop trick to insert the file’s path at the end.
Now set the file’s permissions to the usual defaults:
sudo chmod 755 /path/to/file
The numbers are a shorthand way of telling chmod to set the permissions to “read/write/execute, read/execute only, read/execute only” for the file’s owner, groups and everyone else respectively.
That’s fine for an individual file, but my hard-disk has thousands of them, and who would want to do them all one by one? Fortunately, doing the same for folders of multiple files is just as quick. All you need to do is add the option -R to tell chown or chmod to apply to the contents of the folder recursively, i.e. to all of its files and all the way down through any subfolders:
sudo chown -R username /path/to/folder
sudo chmod -R -N /path/to/folder
sudo chmod -R 755 /path/to/folder
Again, dragging and dropping the folder into Terminal will insert its pathname. If you wanted to speed up applying these to multiple folders, you could move them all temporarily into a single folder and apply the commands to that.
When going through this I encountered a handful of files that couldn’t be changed because they were locked, and the Mac’s Get Info box didn’t give me an option to unlock them (the relevant box was greyed out). But Unix again came to the rescue: to unlock a folder and its contents, the relevant command is
sudo chflags -R nouchg /path/to/folder
These few Unix commands in the Terminal have cleared up all of the file moving/deleting/renaming issues I’d been dealing with. There’s also the question of permissions at the disk level, rather than folders and files on the disk, but I haven’t been able to find a canonical account of what these should be by default for external drives, so I’ve tried not to mess with what’s already there. It hasn’t been a problem so far.
Here are some more resources which helped with different pieces of the puzzle:
Change a user’s User ID on Mac OS X. At first I thought I might need to do this, but ended up not needing to.
What does “@” signify in Unix file permissions?