News for hackers. Stuff that matters.
Re:Why AI Chatbots Agree with You Even When You're Wrong by Kave0ne on Wednesday March 11, 2026 @10:57PM (#47345830)
Fair distinction. You're right that single-turn sycophancy is a training signal problem — memory doesn't fix a model that caves to "I think the answer is X" in the same prompt.

My claim was narrower: for multi-turn drift, architecture matters. AGM belief revision theory (the formal framework for rational belief updates) requires that when new evidence contradicts a prior belief, the revision should minimize change while preserving consistency. A stateless model can't do this — it has no prior to revise against, so agreeableness is literally the rational default. A model that can retrieve "user asserted X last week, then corrected to Y" has actual evidence to weigh the new claim against.

The two failure modes compound: RLHF pushes toward agreeableness, and stateless context removes any countervailing evidence that would justify resistance. Fixing the training signal without fixing the architecture just makes the model slightly less likely to cave — it doesn't give it anything to stand on.

There are no comments on this comment yet.