recovery

Yesterday while freeing space on a number of drives I found that one failed to mount so I had to use the recovery tools in disk utility to get it to mount. The process took a few minutes. Once this was done I decided to move all data from that drive to a safer place. In the process I came across the frustration of backup.backup, or some similarly named folder. It’s the time machine backup folder.

The Problem

The problem with Time Machine backups is that they assume that you want to recover data straight from Time Machine on another mac, rather than directly from the file system. At first I tried to delete program files and other folders that I didn’t want to backup. That failed. I didn’t have permission. I tried to sudo into the drive but that didn’t work either.

The reason for which I wanted to tidy up the backup, to keep just the files I wanted to keep is that there are gigabytes of program files and others, that have no value to me. It’s my personal folder that I wanted to recover. If I could delete all the rubbish surrounding my files I could have just moved the files of value.

I couln’t do this so I went into time machine and noticed that I could mount this old backup from 2009 or so and use it as a current backup disk. Before doing that I backed up my current files.

Once I had backed up my current files I went back and mounted the old drive to the backup tool but couldn’t see the files. I considered running a backup but chose not to because initial backups take hours and I didn’t want to spend hours backing up to a drive that I wanted to remove from circulation.

The drive is only one terabyte, and is an old dinosaur from another age of hard drivess. The noise it made yesterday exhausted me. I was happy to turn it off when I had achieved most of my goals last night.

In Brief

When you’re consolidating drives there is a chance that you find a lot of extrenuous filess such as program files, render files and more. These take up terabytes of space and prolong transfer time. By making file systems as lean as possible, before consolidating files from external drives to a centralised system it pays to remove as much chaff as possible. System files can easily be 80 gigabytes depending on how many programs you install

The Slow Solution

I had two options. I could have copied my user folder for each backup to the secondary drive but that would mean several gigabytes of files with only incremental changes. Instead I went to a full system backup. These usually start with “macintosh HD” rather than “username”. I went in and backed up the pictures, downloads and one or two othr folders from the most recent full backup and oldest full backup.

Ideal Solution One

The first ideal solution is to be able to get admin access to the hard drive and remove all program files, all system files, all users that I am not interested as well as all “noisy” files that I know are either backed up or no longer needed. Ideally I would be left with the pictures folders, and when it’s tidied up I would copy files from 2005 to 2005, 2005 to 2005, and so on, until all files are consolidated in a primary folder, rather spread across secondary folders.

Ideal Solution Two

In theory Time Machine is designed so that you can go to Time Machine and find files within a specific folder over time. This would make backin up all images easy, just copy all, and reject the duplicates. Within 20 cycles you’re up to date and you’re done.

In practice this works once you have synced files and folders between the current mac and the old backup and this takes hours. In this context it’s faster to do things manually, especially since you may lose the very files you’re trying to save as the oldest backup gets squashed by the new files.

Time Machine and Drive Durability

I have been using Time Machine to back files up for several years now and in that time I have had several drives either overheat and fail, before recovering, or failing and requiring a reformat. Yesterday the drive that failed was a Time Machine backup drive. By this I mean a normal drive that I used for backing up, rather than the propietary solution.

Silver Lining

Swapping between Time Machine Backup drives is easy. It takes a few seconds. This gives you the freedom to backup between two or three backup drives. If one drive fails then you can fall back to one or two others and lose a week of data, rather than a few months.

And Finally

Whilst it makes sense to ensure file integrity if you use Time Machine as a backup and recovery solution it is not as useful if you want to recover files, years after you have switched computers and Time Machine Backup drive. That you need to understand file and folder structure to recover files is sub-optimal, especially since you have so little control.

While Time Machine is used as a live backup, it’s okay. Once it is a dormant archive style backup it should be possible to streamline the folder structure for a human to sort through and tidy up a file system.

Conclusion

PhotoPrism

For a long time we could swap ram and hard drives in laptops but recently you are stuck with the drive that you can afford, internally, and rely on external hard drive for storage of photos and other media files. That’s why self-hosted cloud solutions like PhotoPrism and others are so good. With a Pi running PhotoPrism, connected to a four terabyte hard drive you instantly double what you could store with iCloud and other solutions, and you don’t need to delete photos from a local drive, because if you run out of space on the drive connected to a Pi you can upgrade that drive to a higher capacity, over and over.

Nextcloud

The same is true of Nextcloud. With Nextcloud you can download your documents folder to your own self-hosted version of iCloud and you have a backup of files as soon as you create them.

If we use PhotoPrism for personal photo albums then we free up the space that we would have used on our local machines. If we use Nextcloud then our files and folders are backed up every time we save them as they sync to the cloud whenever required. If your laptop drive fails you can recover from the Nextcloud backup, and if you have iCloud running two, then you have a local copy, a local cloud copy and an external cloud copy.

I would consider backing up from the PhotoPrism Originals folder to Kdrive, for cloud backup. iCloud is okay for docs and more but for photos you need up to two terabytes so kDrive is cheaper than iCloud and Google Drive.