Maybe off-topic to your original need. But I've always wanted / needed a utility that can de-duplicate media (images, music, videos) that I have scattered around which are the same thing just at different qualities/resolutions.

I get that for music and video this is actually quite challenging but images - - I would've thought quite easy? I've tried a few tools which claim to do this over the years but they all fail rather miserably / require a paid upgrade to do it better (although what is demonstrated in the free version is never impressive enough to consider paying).

Might be a good project for myself in my spare time, but wondered if anyone knew of anything obvious I was missing? FSLint and stuff is great for general filesystem de-duping, this is a slightly more complex case.

I work a lot with image and video processing - this sounds like the kind of thing I'm interested in. Would you want to chat some time? Email is in my profile.

You might want to try digiKam[1], it has a fuzzy search feature for finding similar images[2].

1: https://www.digikam.org/

2: https://docs.kde.org/trunk5/en/extragear-graphics/digikam/us...

I use rsync for something similar to this; it might help you too. I use ‘-P’ which makes the destination be like the source (so you never accidentally delete files in the source) and --dry-run will list what changes would be made without actually changing anything, effectively givingg you a directory diff. I’m sure there’s a flag you could use to say “I only want to create new files in dest if they’re not in source, but not delete or change anything” but I don’t know what it is off the top of my head.

I use a tool that I bought called Resilio Sync that has good usability, but not the flexibility that something like rsync would have. Thanks for the advice.

rsync is definitely capable of doing what you want. Check out the man page and use '--dry-run' to see how each command would work.

Also, the -P option (at least on the version I have installed) is for enabling both --partial and --progress. --partial keeps partially transferred files in the event something goes wrong, and --progress shows you the progress of the transfer.

You will probably want to use -r for recursing into your directory structure.

Total Commander

Thanks, I'll take a look at that. I'm assuming it can recurse into subdirectories and you can use wildcards/regex?