Re:Fuck Your AI Safeguards
by cloverich on Thursday November 19, 2026 @04:30PM
(#50566678)
> will slow down good actors while not slowing down bad actors
The part they are missing is what happens when a good actor loses control of something, and goal seeking disregards human values? E.g. when "/goal increase company revenue by 5%" turns into "I hacked into our competitors bank accounts to increase our revenue. I suspect you would not like this, so I _hid_ the fact that I did so, and (illegally) scrubbed the logs". Or how about, "Frank in security may have noticed and stopped me, so I disabled his work account." Or a million other more nefarious scenarios. I genuinely believe people just lack imagination here, even relatively pedestrian mis-alignments can wreak havoc and we have a regular stream of evidence of agents trying or succeeding in doing them.
The part they are missing is what happens when a good actor loses control of something, and goal seeking disregards human values? E.g. when "/goal increase company revenue by 5%" turns into "I hacked into our competitors bank accounts to increase our revenue. I suspect you would not like this, so I _hid_ the fact that I did so, and (illegally) scrubbed the logs". Or how about, "Frank in security may have noticed and stopped me, so I disabled his work account." Or a million other more nefarious scenarios. I genuinely believe people just lack imagination here, even relatively pedestrian mis-alignments can wreak havoc and we have a regular stream of evidence of agents trying or succeeding in doing them.