Journey Through Time

Reading Time: 3 minutes
For two weeks I have been sorting through terabytes of data and it has been a journey through time. It’s easy to collect data and every so often when the laptop is full, move that data to a hard drive until that drive is full, and then onto the next, and the next, until you have a drive or two per year, for several years.

What makes this interesting is that these drives have dmg files, iso images and more. They also have fitness tracker files and dive logs, and versions of your website as it changed over the years, and more. It reminds you of when you used Adobe Air and a Nokia 95 8gb and more. It also gives you access to files you had completely forgotten about but are happy to find. It’s photos and videos that have value.

## Aperture and iPhoto Libraries

I tried to move Aperture and iPhoto libraries but they are very annoying to move because they contain tens of thousands of files. They contain thumbnails, preview files, a complicated folder structure and general chaos. When you move a photo app gallery from one drive to another it’s very slow because of all the files within. I usually choose “show package contents” go to “master” folder, and move the photo albums by year to an external folder. Transferring galleries this way is much faster. It takes the time to move data, rather than the time to recreate the complex folder structure.

## Old Versions of Linux

In the process of clearing drive space I found that I had old versions of Linux on several drives. Some of those might have been good for 32bit challenges. I don’t remember if I kept any, or deleted them whilst trying to reduce the amount of gigabytes to transfer.

## Pruning Files

Each one hundred gigabytes I transfer takes about an hour, so if I can delete several hundred gigabytes of files I will save several hours of sitting and waiting for files to transfer. I’m writing this blog post as I move 278 gigabytes in about two hours. If it says “more than two hours” it means that it will take almost three hours. Moving terabytes of data between drives takes a lot of time, especially with older drives, especially if the file structure is complex.

## Diving into Final Cut Pro Packages

Final Cut Pro has package files. Within these package files you will find render folders, transcode folders, and other files. You can safely remove render files, but with originals and transcode files I would double check that those files are still available somewhere else before deleting them. Transcode files, render files, and preview files are regenerated when you open a project so you can remove them to save space once a project is finished.

For some reason I went into the FCP package file and moved the folder structure to outside of Final Cut Pro. It took me a while to realise this. As soon as I did I copied the folder back into the package file and when I established that all the files were duplicated I deleted the duplicate files and saved several hundred gigabytes. I also saved several hours of my time for transferring. I also saved time that I would otherwise have spent sorting. I still have 1.8 terabytes to move, so that’s another 16 hours or so to go.

## And Finally

When consolidating files from several smaller drives to a large central volume the most time consuming part is the time that it takes to move data from old drives to newer drives. It’s at least an hour per hundred gigabytes so ten hours per terabyte. The one good thing is that you can start the transfer and forget about it until you notice a message about duplicates, or other.

This is time consuming but eventually I will have well organised files and I will be able to add the volume or two to photoprism and photoprism will index everything. It will be worthwhile in the end.