ExFat Stability Issues
|

ExFat Stability Issues

Byrichard

Yesterday a drive failed to mount so Photoprism and Audiobookshelf failed to work. The server was up and running but the files were not accessible. For this reason two of my services have been unusable for several hours. I believe that the issue came from using an ExFAT drive rather than EXT4 or another journalled file system.

The reason for which journalling is important is that Pis and other systems crash, and when they crash, read write cycles are not completed. The consequence of this is that the drive that had been fine before the crash becomes corrupted after the crash and needs to be fixed. Most of the time this is quite easy. I use disk utility two to three times, and eventually the problem is fixed long enough to backup the data.

Why This Matters

The rational thing to do is to think, “I’m using these drives for something that NAS drives are designed for so it’s normal that external drives would fail.” The reality is that the file system has a bigger role to play than the drive type. If the drive keeps track of what it was doing, and what was interrupted then it can quickly resume from where it left off. Without a journal the drive just sees missing data and asks for repairs to be done. By “asks for repairs to be done” I mean that it fails to mount until the issue is resolved.

Looking Forward

From now on I will use ExFAT as a temporary solution for when I move data from one OS to another but once the move is finished I will use APFS for mac, EXT4 for Linux systems and NTSF or equivalent for windows when flexibility is not needed. It’s easy to format a drive to a more resilient file system.

And Finally

Once I have backed up the data I will reformat the drive to EXT4 and copy the data back. This should make the drive more resilient.

A Rainy Day

A Rainy Day

Byrichard

This morning it rained and this afternoon I fled from noise pollution. I had planned to write a blog post this morning but inspiration failed to come. I was distracted because I don’t know whether it is the USB ports on a PI 5 that have failed or if it is a drive or two that have failed after being plugged in for several weeks in a row without being turned off. The issue is with external drives.

I moved two hundred and thirty three gigabytes of audiobooks from one drive to another and now I am in the process of moving several years of photos from one drive to another. This drive will then feed audiobookshelf and Nextcloud. I am undecided as to whether it will also feed Photoprism or not. Nextcloud and Audiobookshelf have proved to be the more interesting the apps I have experimented with in recent weeks.

I noticed a fault this morning, which is why I was distracted. For some reason the drive is either failing after being on non-stop for weeks in a row, or the Pi5 has faulty USB ports.I would have tested the drive on a mac to see whether I have the same issue but I am waiting for several hundred gigabytes to be moved from one drive to another. Once the move has been made I can determine what the issue is.

At the moment I am tempted to move Nextcloud and Audiobookshelf to a Pi4 8GB, especially if the USB ports on the Pi5 have failed after a relatively short amount of use. I hope that it is not the drive that has failed because it might be harder to recover some data. Paradoxically I will have lost the three most recent years.

And Finally

It’s because of my experimentation that I have not deleted the photos from Google Photos, Flickr and other places. Once I know that my solutions are reliable I will be destructive about which services I am retiring and which ones I am keeping.

A Short Run After Two Walks with Hiking Sticks
| | |

A Short Run After Two Walks with Hiking Sticks

Byrichard

In the last three days I have been for two walks with hiking sticks and a short run. On Sunday the walk was a ten kilometre loop that took me across several villages. During the walk I noticed that a barn’s roof was generating about 8000 watts of energy despite the day being overcast. If more farmers placed solar panels on their roofs we would have less need for high tension lines across the entirety of Europe ruining natural landscapes as power distribution would be local.

A short Run

I ran just 2.5km before walking the rest of the way. I wasn’t using the 105 CHF On Running cyclon shoes as I need to return those. Instead I was running with the 110 CHF trail glove shoes. I ran a short distance because I haven’t been running for a while so I don’t want to overload my system. I also wanted to make sure that I didn’t feel that my heels had hit the ground too harshly.

Interval Training

I walked for five minutes and then I ran for one minute, and then I ran for one minute, and then I walked for a minute, and then I ran until the bottom of a slope and then I walked up the other side, and then I ran for the flat bit up to the tunnel and then I stopped the running track. I could have been more ambitious but it’s easy to fall into the trap of pushing more than we need to, just for it to look normal or good on strava, Garmin or other places. I am a walker, not a runner. I can walk one and a half hours to two hours per day, every single day, with ease.

I could get to that level of fitness with running but at the moment I need to work on building a good base, and then I can push further. It’s also a way of breaking the walking routine. if I run I can go out for a shorter period of time but get a better cardio workout.

Not Quite Nordic Walking

I started walking with hiking sticks, first because I have a minimal shield if unleashed dogs decide to attack me again, and secondly because it uses my upper body. By the end of the walk on Sunday I could feel that some arm muscles were not used enough in normal walking.

It’s amusing to look at cadence when walking with hiking sticks because for a big portion of the walk my steps per minute was zero. When I use walking sticks I make them long. I take two to three steps between stick movements so it counts as if I am either not walking, or taking a third as many steps as normal. That’s why my step count after a 10km walk is just 8000 for the whole day, rather than 17,000 as it would have been if the step counter had been in a pocket or somewhere else.

And Finally

For the first time in a while I went for a walk without a bag. Usually I always have a bag with me, whether it’s empty, which it usually is, or not. I don’t know whether it impacts my running comfort but I will try to run without a bag for a few runs, to see if I feel a change.

YouTube suggesting Six Videos at a Time
| | |

YouTube suggesting Six Videos at a Time

Byrichard

Yesterday when looking at YouTube over lunch I noticed that they now show six videos at a time, compared to the 20-30 videos they used to show, back in the good old days. This means that you have six videos to choose from. The algorithm is cutting down our choice constantly from 30 videos down to 20. and now 6.

Pigeon Holed

If we watch one channel’s content then that content will be shown exclusively until we grow tired of it, and then we will have to choose from a dozen or less content creators. Sometimes videos will appear in three or four categories. It’s hard to browse when there is so little choice.

Plenty of Choice

In the days of renting DVDs for a night or two we would go into the shop and there would be a few recommendations but then we could go into the library and search for a while, trying to find content by genre, mood or more. Today that browsing experience is getting worse.

When I look for something to watch on YouTube or Netflix and Prime I want to have a real choice. I want to see a breadth of choice within a single screen. I don’t want to be forced to watch what algorithms force people to watch. If you recommend content because it’s popular, but it’s popular because it has been pushed on people, then it is not popular. It is spoon fed. The algorithms are cheating us and content creators. We’re being cheated because we have no choice, and content creators are cheated because they are invisible.

The Paradox

What I liked about YouTube is that it provided us with a breadth of content to choose from. We might have browsed for a few minutes and skimmed through thousands of videos but we had real choice to find ideal content. Now, with six videos being shown at a time we’re forced to pick out of six. This isn’t choice. This is scarcity. Thousands of hours of content are uploaded to youtube every minute and yet the algorithms get everyone to watch the same thing.

The problem is that I don’t know what I want to watch for half an hour to an hour so I don’t have key words that I want to look for. If I’m forced to see six videos, rather than browse, then I’m likely to give up rather than search. Usually we look at YouTube and similar sites to discover new content.

Prime

YouTube wants us to pay for prime but they take our ability to choose. They use algorithms that, because content is pushed on us, become worse and worse with recommendations. We can give feedback, but not proper feedback. I sign up for Prime, enjoy it for a few weeks, and then it becomes toxic and I take a break. The algorithms pigeon hole us, rather than learn about us.

And Finally

I love the medium of video. I love well produced content. I love content of a certain type. Google’s algorithms looks at users and recommends the content that it would give to teenagers to 40 years olds, and vice versa. The recommendation engine knows our age, and our viewing habits over a decade and a half. If people are worried about privacy, just look at ads and YouTube recommendations and you will realise that algorithms know nothing about us. Algorithms, by now, should know that I hate sensationalism. I realise that hating sensationalism is sensationalist. The point is that if recommendations for content are bad, then we are likely to take a break.

Downgrading Icloud from 200 GB to 50 GB
|

Downgrading Icloud from 200 GB to 50 GB

Byrichard

If there was the option to downgrade from 200GB to 100 GB I would have taken it as I needed 51.6 gigabytes to backup my phone and other data. Since I couldn’t I went through and removed my photo backup as well as back ups of folders that don’t need to be backed up because they are cloud serice in the first place. Imagine the rational of backing up backups. It makes sense, but without exageration.

I am able to downgrade because I have Immich, Kdrive, Photoprism and Nextcloud backing up my photos as well as two to three hard drives. The beauty of this solution is that I go from spending 36 CHF per year for iCloud down to 10 CHF per year. I go from spending 100 CHF per year down to about 35 CHF per year, and I go from being obliged to be loyal to cloud providers because I had no local backup of everything, to being free to jump from service to service on a whim. By whim I mean that I can dump a service as soon as a cheaper option appears.

There are two reasons for me dumping iCloud. The first is that they’re more expensive than all other solutions so to use them is to throw away money. The second reason, and the more damning one is that if you store photos on iCloud once you reach a certain library size you’re trapped.

You’re trapped because although you can create several libraries on external drives you can only download photos from the cloud on the system drive, not others. This means that with a 500 GB laptop drive that is full with files you’re trapped within iCloud. Your data is locked in until you get a mac mini, add a two terabyte drive, and download all your files.

In the end I used Immich, Photoprism and Kdrive to backup the data that was trapped in the cloud before removing photo sync from the phone with iCloud, relying on Immich, Photoprism, Nextcloud and Kdrive. Why so many services you may ask. Experimentation, and redundancy. Immich is in very active development so they recommend not to rely on it. Photoprism seems stable. Nextcloud is good too, but it’s better for backing up files from computers and more. Kdrive is the offsite backup.

And Finally

Now that Nextcloud takes care of synching files between devices and Immich, Kdrive and other solutions take care of backing up photos and videos there is less pressure for me to use cloud services such as iCloud. I keep the 50GB plan because I can still backup my phone in the cloud as well as various app data. If it was not for these constraints I could dump iCloud entirely. The biggest storage hog on iCloud is the phone’s backup.

Playing with OpenAudible and Audiobookshelf
|

Playing with OpenAudible and Audiobookshelf

Byrichard

Last night I finished converting all my audiobooks to a DRM free format. In the process I learned that the m4b format renders much faster than mp3. I spent weeks trying to convert AAX files to mp3 and then by accident I agreed to convert files to m4b and it took the time it took for me to walk on my daily walk to be done.

Migrated to Audiobookshelf

Now that all the files have been converted to mp3 and m4b I have uploaded them to Audiobookshelf. Some files failed to import. The rest imported with ease. The advantage that I now have is that I can browse through my audiobook collection faster than if I use either Audible’s phone app and their website. The other advantage is that my files are self-hosted locally so unless my hard drives fail I have my own copy, as I would if I had bought physical books.

Fiddly

When you’re listening to Audiobooks on a laptop the website works well. When you’re listening on an iOS device the epxerience is slightly more complicated. You need to navigate to the website in a browser, find the book, and then start listening. If you use it for podcasts then it plays one podcast and then you need to select the next. On a computer podcasts play one after another.

Create Users

Audiobookshelf allows you to create users. You can have one that has admin privilieges that is used just to add podcasts and create libraries etc and a second one to use as a user. By seperating the two there is less chance of making a mistake.

You can create multiple libraries for multiple users, so in theory you can create a library per person, for them to upload their own books. You cannot restrict library A to user A, library B to user B etc. You can control whether people can upload, download and more. You can give people access to listen, via the site, but restrict who can download to keep.

And Finally

OpenAudible and Audiobookshelf give us a way to keep the content that we spent money buying for our own personal use. If Amazon or Audible go bankrupt then we do not end up with nothing, after spending 100+ usd per year for content. I like both OpenAudible and Audiobookshelf

Using Nextcloud as a Timemachine Backup

Using Nextcloud as a Timemachine Backup

Byrichard

Around a week ago I setup the Nextcloud desktop client to keep an eye on four folders. It synched three out of four folders with ease and struggled with the fourth so I removed it from the sync. Now I have three folders that sync permanently, and when I mean permanently I mean that within seconds of creating two screenshots they were backed up by Nextcloud.

Wireless Backup

By using Nextcloud to backup specific folders I don’t need to have the laptop plugged in to a drive. As soon as a file is created it syncs to the cloud, available on the mobile phone or other devices. You don’t need much storage for this to work. An SD card with 128gb is enough. This isn’t meant to be a secure backup. It’s meant as an iCloud and Google Cloud alternative.

When files are synched as soon as they are created we have the advantage that if the laptop suddenly crashes or fails we have the synched version available within seconds, rather than when we run time machine when the drive is plugged in.

Self-Hosted

Because the device running Nextcloud is self-hosted, either at home or offsight this means that you can backup and recover from anywhere but it also means that if you need a fast recovery you can unplug the drive from the Pi device and backup directly, in an emergency situation. If you have more time then synching from Nextcloud locally will work well.

Audiobookshelf Synchronisation

If you have a collection of Audible or other books that you own and that you want to make available to read via Audiobookshelf you can. You could use OpenAudible to make them DRM free, and from then on get Nextcloud to move them from your personal computer to the server for indexing. This workflow is of interest because it takes time to convert books from one format to another.

I am converting books that I own, as a backup in case Amazon, Audible or other suddenly goes bankrupt and I lose access to books that I own. I use Audiobookshelf because it has good metrics but also because it provided me with hands on experience of setting such a server instance up, and then using it.

And Finally

Once Nextcloud is setup it runs in the background and you don’t need to think about it. It’s easy to install, especially if you use snap install nextcloud. The one thing I would like is to have a simplified method of changing the drive from being local to the system disk to using an external disk.

The Cow and Pheasant
| | |

The Cow and Pheasant

Byrichard

Today I went for my daily walk and I came across a couple of pheasants. One was female and the other was male. I was actually standing right next to the female and didn’t realise until she flew away from me. I was startled but no more. I was more focused on the male pheasant.

A pheasant near cows
A pheasant near cows

I walked closer, to try to get a clearer photo but didn’t succeed. Instead it went into a field with some cows and when one of the cows noticed it went up to investigate. I thought it was chasing the pheasant and eventually it was. It was an amusing sight to see. A cow running after a pheasant.

it got better. When the pheasant went into the next field the rest of the herd came across to look at the pheasant.

A herd of cows looking at a pheasant
A herd of cows looking at a pheasant
On Cancelling On-Running Cyclon membership
| |

On Cancelling On-Running Cyclon membership

Byrichard

On-Running, is ideal, in theory, but sub-optimal in reality. The biggest issue I found with On-Running CloudNeo shoes is that they are seasonal running shoes. If it’s icy or wet you’re going to slip and slide all over the place. If you’re a former snowboarder and cyclist you will recover, but if you’re not used to slipping and sliding you will fall. As a result of this I got shoes a month or two ago but never used them because it was either rainy, or cold.

Imagine having a pair of shoes that you pay 35 CHF per month for, but that can’t be used for 4-6 months per year. That’s a lot of money for shoes that are dormant due to not being well suited to the running environment.

My second grudge with on-running is that they encourage you to think “Oh, you should, but you don’t have to return the shoes that are warn out”. The cost if you don’t return shoes is 100 CHF. If you have warn out shoes with the sole peeling off with the first pair then that is appalling customer service. If a shoe is degrading 100 CHF is very expensive for an unusable pair of shoes.

Comfort

The shoes were comfortable when running, at first but eventually they began to feel like crap. The shoes lasted three months with my use before starting to fall apart. The sole fell off. I felt that the toe box was uncomfortable when wearing them after several weeks of use, especially with walking. I find that on-running shoes in general are not comfortable walking shoes.

Cancellation process

I cancelled my subscription today. I haven’t used the replacement shoes at all, due to the unfriendly weather and badly suited grip for winter Switzerland. As a result I am stuck with a connundrum. Do I return shoes that are perfectly fine, to be recycled, to avoid paying 100 CHF on top of the 35 CHF per month I paid for several months, or do I pay 100 CHF? I think the answer is obvious. Return the shoes.

Idealisticly

Idealisticly, once you cancel the membership you should be given the option of running the shoes until they need to be recycled, and return them then. The first pair I had were worth the 105 CHF I paid. The second pair were worthless due to snow, ice and wet roads making the second pair unusuable for months. They’re still new. I haven’t removed any of the packaging from the shoes yet. I could use them for three months, and then send them to be recycled out of contract. I have paid for them. I just never got to use them due to them not being designed for a Swiss winter, despite being Swiss shoes.

And Finally

I really like the idea of a shoe subscription where shoes are recycled after their useful life is over. I liked running in them and during the summer months they were a pleasure to use, for running. For walking they’re sub-optimal, especially for longer walks.

What I would like with the Cyclon program is an option to suspend the plan while the conditions are not good for the shoes. When it’s raining, snowing, and the ground is frozen these shoes are dangerous. When it’s warm and sunny the shoes are great. Sending them back is easy, and the process is convenient.

In my opinion on-running need to make Cloudneo shoes that are usable in winter, and comfortable for walking, and I will renew my subscription.

Looking through a Seagate Drive at Lightroom
|

Looking through a Seagate Drive at Lightroom

Byrichard

Recently I got a new drive and as I registered it I saw that I could play with lightroom for free for a month. For free, after spending more than ten francs on a hard drive. I have no intention of using Lightroom after the one month Adobe trial period for a simple reason. Paying 10 CHF per month, when you pay for one app, exceptionally, is affordable. Paying 10-20 CHF per app per month becomes exhorbitant.

What did catch my interest is that there is a one terabyte tier for data storage in the cloud. The issue I have with this plan is that it’s 149 CHF per year, when two terabytes with Infomaniak is 67 CHF per year, and 100 CHF per year with Google Photos. That’s 49 CHF more than Google and 80+CHF more than Kdrive. That difference in price doesn’t justify Lightroom’s added functionality.

No Light Room Equivalent for Video

I looked at the Adobe Creative Suite Apps. There is no Media Asset Management tool for video. You have lightroom, for photos. There is a stock footage app but a stock footage app is not a personal Digital Asset Management tool. That’s a nice that Adobe should get into, as it is guaranteed that users of the Adobe Creative Suite needd a solution for photos, and videos.

Video is Supported

Video is supported. You can’t search by videos and there is no mention of video except when you look through assets.

And Finally

I do not plan to be seduced by Adobe Lightroom or other tools. We have plenty of free or open source solutions to pick from. I have Final Cut Pro Studio, Final Cut Pro X, DaVinci Resolve and KDEnlive to play with for video editing, and Immich and PhotoPrism as Lightroom equivalent tools.