You know the article addresses this exact problem? It links to another article focused on this problem, which suggests reducing the saturation of images: https://medium.com/dev-channel/re-colorization-for-dark-mode...

(It doesn't seem to have occurred to that author to try reducing the brightness of the images, or reducing saturation with the saturate() instead of grayscale(), but those also seem worth trying, brightness in particular.)

Apple's Smart Invert is even simpler, it just leaves media like images and video as-is, only inverting colors of other things.

I suppose for diagrams that you know are black-on-white, inversion is nice, but it should be opt-in.