They do have a loss function, that is analogous to human desire.

I think we are fundamentally missing something, that there is an irreconcilable difference between a mathematical expression and actual conscious desire, and until we figure out what they is, we won't crack AGI