Well no, the agent's model of the other person is just data, but this model is so convincing that most people don't perceive it as such and they're thoroughly convinced of the realness of the other person.

Consensus matters because if people around you disagree with you they will kill you or put you in prison. At rock bottom this is how all laws and ethical norms are created.