This isn't theoretical: Google Photos was identifying Black people as gorillas, and they didn't fix it, they worked around the bug by removing "gorilla" as a possible label in 2018. Some here seem to be saying that we can't call this racist unless someone specifically intended to do this because of hate. It's not subjective when someone's own face is so flagged.

https://www.theverge.com/2018/1/12/16882408/google-racist-go...