That probably can be done and may work well already, not sure.
But the harder problem is that since I'm from a country where at least 99% of nurses are white people, then for me it's really natural to expect a picture of a nurse to be a white person by default.
But for a person that's from China, a picture of a nurse is probably expected to be of a chinese person!
But if course the model has no idea who I am.
So, yeah, this seems like a pretty intractable problem to just DWIM. Then again, the whole AI thingie was an intractable problem three years ago, so...
I guess if Google provided the model with the same information if uses to target ads then this would be pretty much achievable.
However, I am not sure I'd like such personalised model. We have enough bubbles already and they don't do much good. From this perspective LLMs are refreshing by treating everyone the same as of now.
Don't forget India, Nigeria, Pakistan, and the Philippines, all of which have more English speakers than any of those countries but the US.
> without some additional prompting or fine tuning that encourages it to do something else.
That tuning has been done for all major current models, I think? Certainly, early image generation models _did_ have issues in this direction.
EDIT: If you think about it, it's clear that this is necessary; a model which only ever produces the average/most likely thing based on its training dataset will produce extremely boring and misleading output (and the problem will compound as its output gets fed into other models...).