Clouds and the Jura in the Late Afternoon

An Alternate Way of Using Nextcloud

Byrichard
Reading Time: 4 minutes

Setting up a Raspberry Pi 4 with 2gb of memory to work as a Nextcloud server is quite easy. Download the right ISO from nextcloudpi.com, flash it, put the card into your pi device and after two or three more steps you have a local machine running Pi but you still need to setup port forwarding, open a UPnP port to access the server externally and other steps.

The simpler solution is to download the Nextcloud app on your phone, as well as for the desktop/laptop that you’re using. I set it up so that any picture I take is automatically synced to a server in Holland. As soon as I take a picture it syncs to the cloud. The images are then synced from the cloud to my local machine, and from there I can archive them either to an external hard drive or another local Nextcloud instance. Once the images are synced I can remove them from the mobile phone, saving money, and reducing the need for an expensive higher capacity phone.

We can use iCloud and Google Photo but both are slow and clunky. They’re great for looking through images, but a nightmare for media asset management. They’re a nightmare because they want to offload your images from your devices, to the cloud, but make recovering images in a timely manner very slow. With Nextcloud image backup becomes a simplified, and quicker process.

With the Tab Digital solution you have several tiers and the first one is 8 gigabytes of data for free. This is more than enough for a day or week of picture taking. You can easily take pictures with the phone, have them sync by wifi, or data plan, to the cloud, and then retrieve them from the web as you get home, to store them on your photo or video hard drives for longer term storage.

With ease you get your photos stored in three locations. Your phone, the cloud, and a local hard drive. If one fails you can resort to the phone, the cloud or a local copy on an external hard drive.

I am toying with using Tab Digital’s instance of Nextcloud to backup my images from the phone while I’m hiking, cycling or doing other sports, having the images sync to a local drive, and then syncing from the local drive to a local version of Nextcloud.

By outsourcing the outward facing server to a professional hosting solution you offload the responsibility of keeping the server secure from attacks. You also keep your private network separate from the open internet.

iCloud Retrieval is Slow

When I tried to recover my photos from iCloud it was slow and painful. You need hundreds of gigabytes of storage for hours or even days. You need to invest days or even weeks in downloading the data and if you’re on a laptop you can’t move the laptop until the backup is over. If the download was fast this would not matter, but it is. The second problem is that iCloud is extremely messy with file names and folders. You might recover your photos but you then spend days or weeks putting things back into good order.

Google Photos

With Google Photos synching from iPhones and Android devices is easy. It’s when you want to recover data that it becomes a chore. When I first tried I could tell it that I wanted to export data, and within a certain amount of time the files would be ready for download. The issue was that because I had several hundred gigabytes it takes time to download so many files, and sometimes they time out so you can only do three or four files at once. Once again it takes days, or weeks to retrieve the data.

Recently I noticed that you can ask for all your images to be transferred from Google Photos to Google Drive, and from Google Drive either keep them there, or transfer them. I had to send them from Google Photos to Google Drive, and from Google Drive to KDrive, and from KDrive I think I will download them.

Nextcloud

With Nextcloud, you go out for the day, and as you take pictures they’re synced to the cloud. Once you get home they’re synced to your home computer, and from there it takes a few seconds to back them up to a personal hard drive. Instead of spending 100 CHF for Google One, 120CHF for iCloud, or several hundred francs for Nextcloud larger storage solution you spend 67 CHF once per year, for two terabytes.

Slow and Painful

As I wrote this post I was playing with the export and import tools from Google to Kdrive and the export of my photos failed. This is the precise reason for which I want to dump Google Drive and iCloud as photo storage solutions. They make it very hard to dump their services, once we find a cheaper solution.

And Finally

Due to the leap from spending three francs per month, to ten francs per month I never upgraded my storage with iCloud. I was against spending 120 CHF per year. I did give in to Google One with two terabytes of storage but I liked the solution only as long as it was cheaper than other options. KDrive then came long for around 67 CHF per year so it’s even cheaper. The problem is that I was trapped with no easy way to transfer my photos from Google Photoes to Google Drive. Google Photos and Google Drive use the same Google One subscription but you can’t download from Google Drive with the photos, until you use takeout to create zips with your photos and tell it to add them to Google Drive.

Once the photos are backup up to Google Drive you can then use the Kdrive tool to import from iCloud, One Cloud, Google Drive and other storage solutions. In so doing you can consolidate all of your files to one remote storage solution. Once this is done you can downgrade Google drive to 200 gigabytes, leave iCloud as is, delete OneCloud and other solutions.

Similar Posts

Immich and iPhone Storage
|

Immich and iPhone Storage

Byrichard

When experimenting with the Immich iPhone app I found it impossible to upload beyond 15,000 images and I supposed that it was because the phone timed out before it had checked all the previous files before moving on to the last four thousand images. In reality the problem is that Immich downloads the media from iCloud and leaves it on the phone. The result is that if you have one hundred gigabytes of photos on iCloud you need one hundred gigabytes of storage ony your phone.

I only came across this after several attempts. I realised that it was a storage issue because another app said “Phone memory is low, please clear some data, or something to this effect. As this is my old phone I could clear files from other apps to make sense for iCloud to download the last of my photos, to be synced with Immich.

A few hours later I had downloaded and synced all iCloud photos to Immich. I can now delete iCloud images and have plenty of space for future growth. I have the 200 gigabyte plan so it doesn’t really matter.

iCloud and iPhones

My second to last phone as an iPhone 8 plus with 256 gigabytes of ram but the next one was an iPhone SE 2 with 128 gigs of ram. The reason for the downgrade in storage is simple. Two hundred gigabytes with Apple costs 3 CHF per month whereas 2 terabytes is 10 CHF per month. The cost per gigabyte makes the 2 terabyte plan more interesting but the difficulty of retrieving data makes the plan uninteresting. You end up paying 120 CHF per year until you find a way to retrieve that data.

The Shift from Self-Storage to Online Storage

Many years ago Google had Picasa, and Apple had and still have iPhoto. Both apps expected you to store your files locally. With time, as people shifted to laptops so the amount of storage available on devices declined. That’s why Picasa enabled cloud storage, and why Google Photos and iPhotos allow you to backup to the cloud and clear space on mobile devices. With android devices it’s easy to add a miniSD card up to two terabytes. With iOS, but also with MacOS devices getting more storage costs hundreds of francs more.

With Apple you pay 1 CHF per month for 50GB, 3 CHF per month for 200GB, 10 CHF for 2TB, 30 CHF for 6TH and 60CHF for 12TB. By dumping my photos from iCloud to Immich I can downgrade my iCloud plan from 200GB for 3CHF per month to 50 GB for a symbolic franc per month.

iCloud Photos use 135 gigabytes of storage out of 185 gigabytes of storage.

Local Storage and Cloud Storage

If you have one hundred and fourty gigabytes of photos then you need a phone with at least 140 gigabytes of storage but the SE has just 128 gigabytes of storage. The laptop has about 256 gigabytes if I remember correctly.

This means that if you store two terabytes of photos in the cloud you need a two terabyte drive to recover them, whether on your phone or your laptop. Since this costs thousands of francs it makes more sense to have a solution such as Immich, Nextcloud and photosync and Photoprism.

I believe that both Nextcloud and Photosync download photos from iCloud, upload them to their respective services and remove them, whereas Immich downloads them and keeps them in place. With Immich you need to have enough storage on your mobile phone whereas the others adapt.

And Finally

Immich did not give an error message. It just got stuck so I tried over and over without success. It’s because another app said that I had run out of space that I was able to resolve the issue and accomplish my goal of transferring files from iPhoto/iCloud to Immich.

Now is the time to evaluate Nextcloud Immich, and Photoprism over a number of days.

Playing with Hard Drives

Playing with Hard Drives

Byrichard

One of the easiest things to do is to buy a hard drive, and over a period of time fill it, and then get another drive, and a third, and a fourth, and a fifth. People will go from a small drive, to a larger drive, and a larger drive after that and backup plenty of the files across three to five drives. The result is that you have terabytes of storage, and some files are “backed up” on every drive, whereas others are precariously stored on just one drive. The result is that if you do want to consolidate that data to a single volume you need a larger and larger drive.

Over the years I have worked as a Media Asset Manager and I have a technique for finding duplicates, and freeing HD space, whilst at the same time ensuring that data is still backed up to at least two volumes. if Not more. As I deal with video I do this by organising every project by year-month-day-country-subject-individual where “individual” is the person working on that project.

Consolidate Media Assets onto a Single Volume

The idea is that as you copy files and folders from a collection of external hard drives you consolidate everything onto a NAS or other form of storage server. If the date is not provided I use the file names and project title and related information to situate it in with time.

Iterate

At first the process is easy, because every project and set of files are unique. The beauty of the year-month-day folder structure is that as you progress you spot that two or three folders have the same name, and that’s when you go and check file dates to see which is the most recent.

Fine tuning

One nuance is that if a file is named final final, and another is named I hope this is the final, and a third is named final final final final, then you change the name. Each file is renamed according to it’s creation date. This allows you to see within seconds which file was last exported, for example, and helps order chaos.

Backing up the Consolidated Files

In an ideal world you would have as many terabytes or petabytes of storage for the NAS but this isn’t always possible. What I did instead was to consolidate the data from the smaller external drives onto the central storage and then store them, as they were, as a backup in case the NAS fails. This isn’t ideal but it’s a good compromise when budget is limited.

Imagine that you have three or four one terabyte drives and they’re all getting full. If you have a two or three terabyte file you may be able to backup one or two drives, but run out of space for the two others. The solution is to get a four or five terabyte hard disk. With this you can dump the four smaller drives, and then start sorting projects and media assets by year-month-day but also by project name. With this work flow you can identify duplicates with ease.

Reducing the Number of Drives Plugged In

The aim of dumping drive A, B, C and D to drive E is that you get rid of the need to have four drives plugged in at once. You can have a projects folder for 2024. Each drive is in its own folder. You open two finder windows and then you can start organising your projects by year-month-day-project-name. You move the folder for the project from the Drive A folder to the year folder. When you finish bringing in drive A you repeat the process with drive B, Drive C and Drive D. When you spot duplicates you delete the duplicate files.

Removing Duplicatew

As you progress you go from having four terabytes of files to having three terabyte, down to two and a half, and so on. As duplicates are eliminated you regain that space so your four terabyte drive is not filled instantly. Ideally you would have the four terabyte drive mirrored on another four terabyte drive. If space is tight then you can re-use the small drives. 2024 can then be split across two or three drives. Now you have a primary storage solution, and several secondary drives. If the four terabyte drive fails you either fail over to the clone, or you fail over to the re-used smaller capacity drives.

If the four terabyte volume fails then you get a new four terabyte drive, and copy the data from the four backup drives and within a short amount of time you have recovered.

Shifting to Exfat

For a long time I was using MacOS so I had drives that were formatted with APFS or the Journaled versions. As I shifted from MacOS to Windows and Linux I found the need to have drives that could be read by all three systems. By consolidating media I gave myself the room I needed to backup a drive from one volume to another, convert it to exfat, and then move data back.

If the windows machine fails, or the Linux machine fails, or the Windows machine fails, then I do not want to have to replace that machine, just because of a file system. By having Exfat I have the freedom to slide from machine to machine, with fluidity.

And Finally

It can be overwhelming to see that you have ten to twenty hard drives that may contain duplicates. By getting a larger drive you can go from having a dozen drives, and a dozen places where things are stored to having a single place where everything is centralised. Once everything is centralised you can order files in folders that are organised by year, and then by year-month-day. In so doing you don’t need to know what you’re archiving intimately. You just need to be able to organise things by year-month-day.

I know that I repeat this point a lot but there is a reason. If you know when something was photographed, filmed or worked on then you can find files within seconds, rather than hours, and without the CMS, should the CMS fail. The other reason is that a project might have two or three names, depending on who worked on the project. If it’s organised well, then going upstream is easier.

This should be an itterative process. Start by the task that is easiest, and then itterate and fine tune the files and folders until everything is well organised, duplicates are detected and consolidated, or deleted, dependant on context. If you do this well then it can be relaxing.

I’ve been consolidating files and data to free up drives to experiment with Immich, PhotoPrism and Nextcloud in parallel.

| |

Travian, an online game

Byrichard

Travian is one of those games you would have loved to play after reading a few Asterix comics. There are three civilisations you can be, Gaul, Roman or Teuton. Each player starts off with one village and expands from there.

There are four types of resources. These are wood, clay, iron and wheat. As your village begins you need to decide on what resources you want to grow. Click on the land type and you will see a display with the amount of time it will take for the order to be completed. The next step is to look at your town. Build a granery, a cranny, a warehouse and other buildings that make a village what it is. Overtime as your buildings improve so the town is large enough for more buildings. These are stables, palaces, residences for kings and more.

Getting your village to expand is just one step. The next step is to make sure that you’re safe. Around you are many other villages and some of these players have been playing for a number of months. As a result they have amassed many resources and alliances. They will attack and pillage you therefore you must defend your land with troops. As you expand and form alliances so the game becomes more interesting. At this point you are competing not just as a village but as alliances to see which can be the most prestigious.

It’s a fun game, reminiscient of such games as Civilization amongst others. If you need a two minute break whilst working on a long project then this may be the game for you. I’ve enjoyed it and so may you.

|

Social media and Interactivity

Byrichard

I have been studying and discussing media tech and society. I have also been listening to the podcasts from the SXSW event on interactivity and my view of technology and daily life has changed. That’s reflected by what I was going to post as an answer to a comment on flickr.

The iphone will sell because it’s what people want. People have accounts on flickr, they’re part of myspace and they’re on facebook. There friends are as well.

Recently I’ve been going out with a camera and taking up to 180 pictures in one night. I upload them to facebook, controlling who can see them and they’re popular. People get round to requesting more pictures so I need to take more.

With my mobile phone comes a one gig mini sd card therefore I can store thousands of pictures. I show people the pictures and if they want to see them online then at least I’ve got a practical way of getting people to approve them.

What was SXSW about? Interactivity. Why? because social networking websites encourage the free sharing of all the media you’re creating.

|

Touch typing on the N95 8GB

Byrichard

Whilst at a number of concerts I had to type fast, very fast on a mobile phone keyboard. Within a very short time I began touch typing. I can type whilst looking around. That’s a useful feature of certain phones. It’s useful because it means the keyboard is well designed.

Are there many mobile devices that you use and can you touch type on them. Do you touch type on your blackberries, iphones or other devices or do you see that as the preserve of computer use?

Playing with Nextcloud Ambitiously and Crashing Immich

Playing with Nextcloud Ambitiously and Crashing Immich

Byrichard

Last night I decided to play with Nextcloud more ambitiously. I set up four synchronisations, one for documents, one for photos, one for downloads and one for desktop. I did it this way because if I went to my root user account and backed everything up I would backup files that are not critical.

It makes sense to backup the documents, downloads and desktop folders because that’s where the most useful things are. In theory I should backup photos too, and I did set that with one Pi but not the other. The reason is simple. Photos are already backed up by Immich, PhotoPrism and Kdrive, so to backup that folder makes little sense, especially since photos are usually taken with the phone, so that’s where the backup should take place.

Twin Experiment with Pi4 and Pi5

On the topic of Immich I do have Immich and Nextcloud running on a Pi4 and on a Pi5. The Pi4 has either crashed, or slowed down to a crawl due to overheating several times, wheras the Pi5 seems to be coping with my backing up of files from the laptop I am using to Nextcloud.

My goal of running the Pi4 with Immich and Nextcloud was mainly to see if Immich worked faster than Photoprism but so far that is not the case. Immich appears to be crashing or slowing down the Pi4 to a crawl consistently. Photoprism on the Pi5 is stable but slow. For weeks, or even months I have been waiting for Photoprism to finish indexing all of the photos that are on that system.

New Files Show Up on Immich

On the positive side at least new files that are uploaded to Immich show up, so even if it takes a while for the back catalogue to be archived the current images and videos do show up and are playable, in the brief windows when the system has not crashed. I am of the impression that these crashes are due to using the desktop version on a Pi4, rather than because of Nextcloud and Immich.

External Volume Not Showing Up on Nextcloud

I used the plugin to allow for external volumes to be used from within Nextcloud but they are not visible when using the desktop sync tool. I will setup a 1tb SD card to run Nextcloud from the Pi4 with a server version of Raspbian or Ubuntu and see how well that works. The desktop is nice to have, but not essential, especially if it is the source of crashes.

Replacing the Wired Time Machine Backup

I am currently using two old drives as Time Machine backups. They need to be plugged in either every day, or on alternative days to backup the system. The issue is that when they are plugged in I cannot move the machine until I eject the disks. I use Nextcloud to sync files that change regularly. Time Machine then becomes a secondary backup tool.

And Finally

For two or three months I have been playing with backing up photos and videos from the phone to photoprism and Kdrive and those systems work well. What I like about photoprism is that it backs up in the evening when I am sleeping. Kdrive backs up almost as soon as I take photos. Kdrive also has the advantage of allowing me to retrieve photos with ease. they’re organised by year and month so moving them around is very easy.

With Nextcloud backing up document files, desktop files and downloads I am synching/backing up files constantly.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.