But, of course, since race is a sensitive topic, we think that this specific detail is impossible for it to answer correctly. "Correct" in this context is whatever makes sense based on the data it was trained on. When faced with an ambiguous prompt, it should cycle through the most accurate answers, but it shouldn't hallucinate data that doesn't exist.
The only issue here is that it clearly generates wrong results from a historical standpoint, i.e. it's a hallucination. A prompt might also ask it to generate incoherent results anyway, but that shouldn't be the default result.
If I ask it to generate an image of a "person", surely it understands what I mean based on its training data. So the output should fit the description of "person", but it should be free to choose every other detail _also_ based on its training data. So it should make a decision about the person's sex, skin color, hair color, eye color, etc., just as it should decide about the background, and anything else in the image. That is, when faced with ambiguity, it should make a _plausible_ decision.
But it _definitely_ shouldn't show me a person with purple skin color and no eyes, because that's not based in reality[1], unless I specifically ask it to.
If the technology can't give us these assurances, then it's clearly an issue that should be resolved. I'm not an AI engineer, so it's out of my wheelhouse to say how.
[1]: Or, at the very least, there have been very few people that match that description, so there should be a very small chance for it to produce such output.
It seems obvious to me that this is just not a problem that is solvable and the AI companies are going to have to find a way to justify the public why they're not going to play this game, otherwise they are going to tie themselves up in knots.