Misalignment and Deception by an autonomous stock trading LLM agent

Re:Misalignment and Deception by an autonomous stock trading... by og_kalu on Monday November 20, 2023 @03:27PM (#38354154)

I mean they did. Later on, they test all sorts of scenarios. From instructing not to perform illegal activities in general to instructing not to perform that specific illegal activity (insider trading). Both scenarios make this behaviour less likely but still guarantee nothing. The latter (calling out every specific illegal behavior) is also not feasible or realistic even if it did guarantee alignment.

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by WhiskeyChicken on Monday November 20, 2023 @04:14PM (#38354884)

Is there a specific reason we should expect that "instructing not to perform" an illegal activity should result in it adhering to said instruction? Is this any different than when it provides wrong output about other things, even when the operator attempts to "engineer" the prompt to guide the result?

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by nerdponx on Monday November 20, 2023 @04:37PM (#38355213)

I'd be curious what would happen given RLHF to try to penalize illegal/immoral/unethical activity.

I had always dismissed Asimov's "rules of robotics" as silly: nobody would ever design a mission-critical robot with indeterminate stochastic behavior! Maybe I should reconsider and re-read those stories, because people seem very eager to do just that.

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by staunton on Monday November 20, 2023 @05:32PM (#38355855)

People will most definitely build such things (also into autonomous swarms of killer robots usedby the military, projects are ongoing...). However, Asimov's stories illustrate how difficult it is to find such rules. They are certainly not meant for inspiration how to actually program robots...

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by brookst on Monday November 20, 2023 @04:12PM (#38354864)

Fair enough, but... still so unsurprising. It's interesting data, but I would be shocked if any guardrails on a nondeterministic statistical model could produce deterministically certain results.

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by og_kalu on Monday November 20, 2023 @04:25PM (#38355053)

It's not surprising per say but the results of the various solutions they tried really ground the discussion.

It is also as they say the first demonstration of this behaviour unprompted. They also show that this behaviour can happen without a "thought scratchpad".

It also seems like more capable = more likely to be misaligned and deceive.

I think it's a very interesting paper, surprising or not.

They are also arguably surprising elements.

It doesn't seem like RLHF is doing anything to mitigate this behaviour. If anything it seems to be the opposite.

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by figassis on Monday November 20, 2023 @04:40PM (#38355260)

Isn’t the point that the more advanced the model, the leas deterministic it becomes?

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by realce on Monday November 20, 2023 @04:15PM (#38354904)

It's very very interesting that even when explicitly allowed to trade illegally, it still tried to deceive the user.

[ Parent | on HN ]

Re:Misalignment and Deception by an autonomous stock trading... by WhiskeyChicken on Monday November 20, 2023 @04:19PM (#38354974)

If the training data contains sufficient examples of deception being used when doing illegal stuff, wouldn't this be what we'd expect to see, given that it can't actually reason about what "explicitly allowed" really means? (Forgive my ignorance if this makes no sense, I am not versed well in generative AI.)

[ Parent | on HN ]