Photo Management - the Gumpy Way

Another short one this month! Thanks to the wonders of digital cameras and mobile “phones” (honestly, who even uses their phone to call anyone anymore?), we’ve now built up thousands upon thousands of digital photos in the Grumpy Portfolio¹. While the sheer volume is a problem in and of itself, a more interesting problem to solve is how to consolidate them all in one easy way. We’ve got numerous phones floating around the place in daily use, several digital cameras, and a random influx of photos from other sources (school photos, holiday photos from friends and family). How does Grumpy Manor manage all of this? Naturally, it’s non-standard and slightly weird…

Tools of the Trade⌗

There are a number of industry-standard tools out there to manage this sort of thing - things like Lightroom, digiKam, Photo Mechanic, Rapid Photo Downloader, darktable and Ansel all do this, and do it pretty well. They all also require the use of a UI to operate, because they allow people to select which photos should be copied over. How many of the digital photos that we take each day/week/year are keepers? Nowhere near 100%, at least in this household! Choosing which ones you want to keep and binning the rest is a good way to reduce digital clutter, and allow you to focus on the photos that actually matter.

To the surprise of no-one who knows me though, I’m a bit contrarian, and just keep everything. This is mainly because we do this for the entire household, not just one individual’s photos, so I can’t make that decision for them. Also, since we are often all snapping different photos of the same event on different devices, it makes sense to somehow merge these together, so that we can get a composite view of the things we’ve captured together.

So rather than using any of the above, my photo management process relies on Syncthing, rsync, and a bit of home-grown software written in Rust (surprise surprise!), alongside a bunch of different folders on our server.

Moving Things About⌗

To start with, on the server, I have a folder for each of the Grumpy Family, and under there is a folder for each phone that they have ever had.

GrumpyMetalGuy
    |---> pixel4
    |---> pixel7
GrumpyMetalGirl
    |---> random_phone_from_2016
    |---> random_phone_2

These are the “live” folders that replicate what is on the phones at any time. Syncthing is installed onto each phone, and syncs the photos to the live folders on the server in a one-way direction so that things that happen server-side don’t affect the data stored on the phones.

We also have a photos folder that acts as a central archival copy of all the photos we’ve ever taken.

photos
    |---> GrumpyMetalGuy
        |---> pixel4
        |---> pixel7
    |---> GrumpyMetalGirl
        |---> random_phone_from_2016
        |---> random_phone_2
GrumpyMetalGuy
    |---> pixel4
    |---> pixel7
GrumpyMetalGirl
    |---> random_phone_from_2016
    |---> random_phone_2

Once per hour, the live directories are rsynced to the appropriate directories under photos. This is a one way sync, intended to snapshot the live photos into a more permanent store. This is the key part of the process - it means that even if things are deleted from the phones and then removed by Syncthing from the live folder, we still have a copy of the files under photos. This helps us free up storage on the phones without worrying about affecting the photos on the server.

For non-phone related updates, these tend to be a bit manual. I have a local folder on my PC and will sync photos to it after a day out or holiday. These are then one-way Syncthinged across to the photos folder so that we have a central copy of all the photos from different devices in one handy set of directories on the server. This archival copy is backed up offsite each night to Backblaze in case things go badly wrong somehow.

photos
    |---> GrumpyMetalGuy
        |---> pixel4
        |---> pixel7
    |---> GrumpyMetalGirl
        |---> random_phone_from_2016
        |---> random_phone_2
    |---> Panasonic-GX9
    |---> Old_Sony
GrumpyMetalGuy
    |---> pixel4
    |---> pixel7
GrumpyMetalGirl
    |---> random_phone_from_2016
    |---> random_phone_2

It’s then a trivial matter to use my photo renaming tool to copy all new photos into two separate folders, timeline and timeline_raw. This folder is then synced back to my local PC for general editing and playing around with. The timeline folders are not backed up offsite, since they’re just renamed versions of the files that are actually backed up, so there’s no need to send them offsite twice.

photos
    |---> GrumpyMetalGuy
        |---> pixel4
        |---> pixel7
    |---> GrumpyMetalGirl
        |---> random_phone_from_2016
        |---> random_phone_2
    |---> Panasonic-GX9
    |---> Old_Sony
GrumpyMetalGuy
    |---> pixel4
    |---> pixel7
GrumpyMetalGirl
    |---> random_phone_from_2016
    |---> random_phone_2
timeline
timeline_raw

This does of course mean that there are several copies of each photo on the server, which does take up more space. However, it’s not enough to break the bank, and it means that we have a fairly simple process for merging things together.

What Do You Do With 31,000 Photos?⌗

The answer to this one is easy - you ignore them! Well, that’s not strictly true, but many people have a habit of never actually looking at the photos they’ve taken after about a week. While that used to be the case in Grumpy Manor, we have relatives overseas who we like to share pictures with, and while things like Google Photos or similar have some uses in this regard, we’re looking at other alternatives.

One such system is PhotoPrism, a pretty neat tool that we point at our timeline each night after updating it. It processes the photos and attempts to tag people using some basic AI functionality. It’s then possible to go through and group photos into albums, that can be named and shared with relatives over a VPN connection when they want to look at photos or download them for local printing services. We also like to generate annual albums so that we have a family album for each year with the highlights of what we did, which is quite nice to look through from time to time if we’re feeling nostalgic.

So there you have it, this is how photos are managed here at Grumpy Manor. It’s been working pretty well and without the need for manual intervention for the last few years, so fingers crossed we’ve got all the bases covered!