Migrating Photoprism From One Machine to Another

Reading Time: 4 minutes

Due to the Raspberry Pi 5, and older, having issues with heat throttling and more it makes sense to build a Photoprism on a "normal" laptop before migrating towards the Pi. The process is an interesting one.

Photo Consolidation

The First step is to consolidate your photos from Google Photos, Apple Photos, Flick and any other source you might have. The simplest method is to organise them chronologically, and then to spend time removing as many duplicates as possible. There are tools for that. They will help you add exif data back into the photo exif fields, as well as look for duplicates.

Photo Ingestion

If your library is organised and ready there are three folders that are interesting to us.

./photoprism/originals:/photoprism/originals

./photoprism/import:/photoprism/import

./photoprism/storage:/photoprism/storage

Originals

The originals folder is where you put your chronological library. Photoprism will automatically index all the files in this folder. If you have decades of photos this takes time. It applies machine learning to catalogue dates, locations, people, objects and more.

Storage

The storage folder is critical because this is where json, yaml and other metadata files. This is also where the thumbnails are stored and this is key for migrating from a high spec laptop to a limited spec Pi. When I migrated this folder I had 1.8 million files for 110,000 photos.

Import

The import folder works as a stepping stone. It allows you to import photos and videos at a later date. If you select "move" then it will import photos, and then move them out of the folder leaving it empty.

Phone Ingestion via Photosync

Photosync is a partially free app. If you pay once you can unlock more functionality. With Photosync you can setup photoprism syncing. You add a configuration title, for example "laptop" and then you select the destination folder, and make sure to use "create sub-directories yyyy/mm/dd to preserve the hierarchy you spent hours preparing earlier.

Be careful, because usually the default is Device Name + Album Name. There are other options but they are out of scope for now.

The advantage of Photosync is that once it is set up correctly, it will keep uploading photos according to the hierarchy you want, eliminating the need to do things manually.

Moving to the Pi 5

Once all the heavy lifting has been done, and the logs say that tasks are completed you can move everything to an NVMe card or external hard drive depending on budget, and convenience. With an NVMe card and a 400 gb library you still have head room. With a 2-6TB hard drive you have plenty of head room.

It’s worth keeping in mind that the docker-compose file for an intel machine and a Pi are different so you will need to find a template for the Pi and ARM architecture.

I like to have a folder /apps/photoprism/ for the docker compose file on the SD card. I also have /apps/photoprism/database on the HD, that was prepared on the laptop, as well as /photoprism/photos, photoprism/storage and photoprism/import on the external hard drive. I prefer to keep the photos and DB separated.

The Thumbnail Mistake

When I migrated from the laptop to the Pi I didn’t bother to copy the storage folder because I thought "the thumbnails won’t take too long to generate". I quickly realised the error of my ways, shut down the Pi, rsynced the files, and then plugged the drive back in before rebooting the file. Within seconds Photoprism was happy.

If I had let the Pi regenerate all of the thumbnails and video files it would take hours, if not days and weeks.

The Vanished HEIC Files

According to the laptop library I have 119,000 photos but according to the Pi5 I only have 110,000. That’s a huge difference. I noticed that the photos that were missing were those taken with a phone, specifically HEIC files. I tried uploading them as jpeg and they appeared almost instantly.

The quick solution is to upload the photos as jpeg rather than HEIC via Photosync. The slow solution, and this isn’t a solution, since it makes the Pi unusable for hours is to reindex the library. I am attempting this now.

The Thumbnails Exist

the paradox, in this situation, is that Photoprism, on the intel based laptop already did all of the heavy lifting so Photoprism should just have seen that each photo had a thumbnail and reflected that in the index, instead of re-inventing the wheel.

And Finally

The Good

I was able to migrate my library from one computer to the other and it worked almost flawlessly except for the HEIC issue. If I had transferred jpeg images I would have had a flawless migration.

The Mistake

The mistake is to re-index the entire library, especially in light of the limitations of rRaspberry Pi devices.

The elegant Solution

The elegant solution is to revive an old phone with all these photos, make sure Photosync is installed and setup, and tell it to upload to the Pi5 as JPEG rather than HEIC. It takes more than five minutes, but it worked instantly with a few test photos.

Silver Lining

The Proof of Concept migration was a success. If I had moved between two intel devices it would have been flawless. It’s because I migrated from one architecture to another that I hit a little snag. I simply noticed it while writing the blog post.