bigmack32

Let me know how it works!

It uses a lightweight ML model to calculate image similarity (cosine similarity of the image embeddings), so it will work on reencoded jpegs and such. The similarity threshold for detecting duplicates is configurable, so you can play around with it if the default doesn't work for you.

It also displays file size & dimensions when reviewing duplicates, but doesn't use that to determine similarity.

I built a tool that allows you to review and delete duplicate photos: https://github.com/mtalcott/google-photos-deduper

I used it on my own library of ~70k photos to delete ~20k duplicates. Hope it can work for others too. There seems to be interest in this feature from Google Photos users, but it hasn't ever made its way into the product.