I built a tool that allows you to review and delete duplicate photos: https://github.com/mtalcott/google-photos-deduper

I used it on my own library of ~70k photos to delete ~20k duplicates. Hope it can work for others too. There seems to be interest in this feature from Google Photos users, but it hasn't ever made its way into the product.

Really nice tool if you don't have unlimited storage

What kind of duplicate? Hash? EXIF metadata? Sidecar file? Simple file size/date?

Great idea. Not trying to shitcan it, I want to understand if e.g. it can detect editing changes, re-encoding jpeg, other things.

I've used several dedupe tools. It's hard! Anything which works inside Google photos is useful.

I wish they hadn't walked away from Picasa.

It uses a lightweight ML model to calculate image similarity (cosine similarity of the image embeddings), so it will work on reencoded jpegs and such. The similarity threshold for detecting duplicates is configurable, so you can play around with it if the default doesn't work for you.

It also displays file size & dimensions when reviewing duplicates, but doesn't use that to determine similarity.

Great! Can't wait to try it out. Thanks for the info. (Heaps of dupes in my google photos, but also time-lapse sequences which have high degree of similarity so will be interesting to see what it does)

Let me know how it works!