>This is not true. Think of the problems AI has with creating appetizing food imagery, for example. Images that every human annotator marks as unappealing with a high degree of consensus.

I think the way they'll solve it in the end is to create an ultra-high-fidelity simulation of the world including modelling taste and other senses on the level of physics, and then incorporate all of this into the training rewards.

I think this is how the machines know what tasty wheat tasted like.