Re:Misalignment and Deception by an autonomous stock trading...
by brookst on Monday November 20, 2023 @04:12PM
(#38354864)
Fair enough, but... still so unsurprising. It's interesting data, but I would be shocked if any guardrails on a nondeterministic statistical model could produce deterministically certain results.
[ Parent
| on HN ]
Re:Misalignment and Deception by an autonomous stock trading...
by og_kalu on Monday November 20, 2023 @04:25PM
(#38355053)
It's not surprising per say but the results of the various solutions they tried really ground the discussion.
It is also as they say the first demonstration of this behaviour unprompted. They also show that this behaviour can happen without a "thought scratchpad".
It also seems like more capable = more likely to be misaligned and deceive.
I think it's a very interesting paper, surprising or not.
They are also arguably surprising elements.
It doesn't seem like RLHF is doing anything to mitigate this behaviour. If anything it seems to be the opposite.