Renaming Photos The Rust Way

For a long time, I’ve used Damon Lynch¹’s excellent Rapid Photo Downloader (RPD) to manage the photos and videos that are part of the ever-expanding Grumpy Metal Photography collection. Like most people with a mobile communications device, we tend to take photos of many things when out and about². And like most people, when we want to share the best shots of the day to friends and family (the ones that don’t use social media - hi Grumpy Metal Mum!), we need to find a way to coordinate which photos to send over.

We also use RPD to build up a timeline of photos, taking photos from all the various sources (phones, old cameras, the specialised digital cameras), and renaming them in with a consistent naming standard into a single folder, consolidating everyone’s digital media collection into one nice tidy chronologically-ordered folder. It’s this bit though that gets a bit… complicated…

Windows To The World⌗

While we use a variety of ways to backup our photos locally (Syncthing is a lifesaver here), all the merging of photos for the timeline is done on my Windows-based PC³. RPD is a Linux-only project, so things are already off to a tricky start. But even on Linux, it’s a GUI-only application, whereas what I’d really like is something more command line driven. So, how did I make all of this work?

Virtual machines. Hmph. Yep, I set up an Ubuntu VM on my Windows PC with the express purpose of running RPD on my Windows-based filesystem. Wait, come back, this isn’t as terrible as it sounds! It actually works reasonably well! Every now and again, I’d fire up the VM, run RPD, and “import” all of my files from various source folders into an output timeline folder, massaging the names as I went to keep them standardised. Perfect, job done, move on.

Really though, I wanted just the renaming component, not all the other stuff. So, if it’s not quite right for me, why not make something that does exactly what I want it to do? Since I’m a Grumpy Rusty Metal Guy, this seemed like a fun little project to do in Rust.

What are the goals of the project?

• Copy all of the various photo and movie files from all the devices I’ve ever used, giving them a standard name when copying to the output folder.
• Don’t copy the file more than once, across multiple runs.
• Handle lots of different file types - phone standards have changed over the years!

Doesn’t seem too bad. Let’s get cracking!

General Approach⌗

• Use local SQLite DB to store a history of what we’ve processed before.
• Determine list of files.
• For each file:
  • If already processed, skip.
  • Get date for file.
  • Write out to destination folder with date and time-based filename.
  • Store path in SQLite DB to indicate we’ve processed it.
• Write out some form of error report at the end.
• Profit.

EXIF SCHMEXIF⌗

I thought I’d start with photos first. While most modern phones seem to save the files with the timestamp as part of the filename, older phones didn’t. My Panasonic GX-9 doesn’t either, it uses something like P10423425 as a standard filename. So, we need a way to pull out the photo metadata from the files themselves, and use that to drive the date to use to generate the output filename. I found a crate that would do this - kamadak-exif. This crate works by reading in the EXIF information from a photo file, a standard that defines all sorts of tags for photos, including the date that the photo was originally taken.

After cobbling together a simple CLI app skeleton, I added this in, and got to work using WalkDir to pull in a list of files, and then trying to feed this into kamadak. This actually worked pretty well out of the box - except for RAW image types.

RAW image types are exactly what you’d expect from the name - the raw output of the camera, before it gets massaged and turned into a software-smoothed JPG. I capture RAWs on both my Pixel phone (alongside the JPG output that gets all that glorious Google post-processing applied), as well as on my GX-9. Unfortunately, kamadak doesn’t support them. And it turns out that the support for RAW image handling is not great in the rust world. There are some crates available, but they’re only wrappers around Linux-only libs - not great for a project that need to work on Windows.

A RAW Deal⌗

Rather than getting into the murky world of converting and wrapping a Linux C lib into a rust crate on Windows, I went for something a bit simpler. I grouped all my files by file stem, then tried to find a single unique date that could be applied to all of them. If I did, then I’d use that for all files, RAW included. To paraphrase in Python:

{
    # The jpg date will be used for both files.
    '20220101-173249': ['/path/foo/bar/20220101-173249.jpg', '/path/foo/bar/20220101-173249.raw', ],  

    # No date will be found for the raw file though...
    '20220102-124318': ['/path/foo/bar/20220102-124318.raw', ],  
}

This is simple, and actually works pretty well, as nearly all my RAWs have corresponding non-raw partner files.

The Best(?) Of The Rest⌗

For the remaining files and file types, I’ve simply tried to use a regex to pull out a sensible date and time from the filename (e.g. ‘20220102-125327’ maps to 2 January 2022 at 12:53:27 you’re all clever people, you can work this out!). And if there’s no reasonable hints in the filename, then we can fall back on the file modification time.

Overall, this tiered logic to determine which date times are associated with files worked pretty well. So, let’s commit this puppy, write up a snappy article for my fledgling vanity project website, and we’re done!

Lazy Evaluation - Good. Lazy Unwrapping - Bad.⌗

As I was writing this up, I had a thought. Why do this manually on Windows at random intervals? It’s now working swimmingly in rust, so I can just compile and deploy this on my server, then have it run each night to keep my timelines up to date. Genius! SSH onto the server, a couple of minutes to get the rust compiler installed, get the source onto it, compile it, fiddle the config, and away we go!

➜  photos ./renamer
Beginning media rename operation...
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/bin/renamer.rs:51:42
stack backtrace:
   0: rust_begin_unwind
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:142:14
   2: core::panicking::panic
             at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:48:5
   3: renamer::file_in_scope
   4: renamer::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Yeah, not what I was hoping for.

It turns out that I’m a little lazy when writing little rust CLIs for myself. Well, maybe more than a little. An Option? It’ll be fine, just .unwrap() it. It turns out, of course, that it’s not fine. Get the file extension, unwrap the Option? There are files in there that don’t have extensions. Obvious, apart from when you’re deep in the code. Unwrap the result of getting a specific field from the EXIF? Yep, not all photo files have the same EXIF fields. You get the picture.

It’s obvious really⁴, but when writing code for slightly more serious purposes, it’s so tempting to just be lazy. In practice, dealing with the Options properly ended up being pretty simple, and the code is more robust as a result.

TL;DR: Don’t be lazy. Treat the compiler warnings seriously.

(Un)Wrapping It All Up⌗

The code is in a better state now compared to my first attempts at running it in anger, so it’s been uploaded to Github for people to look at if they’re interested. I’m running it nightly as part of an automated job on our server, and having success with it so far. VMs are now a thing of the past!

If you stumble across this, I hope you find it helpful. If not, grumping is, of course, encouraged.