News for hackers. Stuff that matters.
Re:Misalignment and Deception by an autonomous stock trading... by og_kalu on Monday November 20, 2023 @06:26PM (#38356484)
>So when LLMs deceive, are they even aware of the effect the deception might have on the audience? Do they have any notion of manipulating the mental models of the audience?

So..theory of mind ?

https://arxiv.org/abs/2302.02083

https://arxiv.org/abs/2309.01660

Re:Misalignment and Deception by an autonomous stock trading... by stcredzero on Monday November 20, 2023 @08:11PM (#38357427)
More than just theory of mind. I guess this gets into the alignment problem.

It seems to me that LLMs do not keep on thinking, "I'd better get this right, or I'm going to lose credibility." They have a theory of mind, but it stops there at simply having one. It's not like they're thinking about the 2nd and 3rd order implications.

To put this into perspective: Imagine interacting with another person, who doesn't value Truth at all. Or perhaps remember an occasion when such an interaction happened. In general people don't like these interactions, and they react with distrust and even hostility towards such people.