News for hackers. Stuff that matters.
Re:The Waluigi Effect: an explanation of bizarre semiotic ef... by Imnimo on Monday March 06, 2023 @01:46PM (#35045674)
>Now, tbh it does rely on the assumption that we are actually seeing more undesired behaviour than before. If that was false then it would falsify the Waluigi hypothesis.

This is exactly my point. There is no evidence given that we are seeing more Waluiginess post-RLHF than we did pre-RLHF. The competing hypothesis seeks to explain the behavior we actually have evidence for, which is "it is disappointingly easy to elicit undesirable behavior from a model after RLHF". The proposed explanation is "maybe it was also easy to elicit before RLHF". If we believe the author's claim that Luigis and Waluigis have "high K-complexity" (this is an abuse of the concept of Kolmogorov complexity, but we'll roll with it), the explanation that Luigis and Waluigis come from the part of training with lots of dense information rather than the part with a little sparse information is far more parsimonious.

Re:The Waluigi Effect: an explanation of bizarre semiotic ef... by jefftk on Monday March 06, 2023 @04:17PM (#35047735)
> There is no evidence given that we are seeing more Waluiginess post-RLHF than we did pre-RLHF.

Testing with the non-RLHF GPT 3.5 API you could probably figure out whether there's more or less Waluiginess, but you're right they post doesn't present this.

Re:The Waluigi Effect: an explanation of bizarre semiotic ef... by dragonwriter on Monday March 06, 2023 @04:45PM (#35048101)
> Testing with the non-RLHF GPT 3.5 API

There is no such API, though, is there? AFAIK, GPT-3.5-turbo, either the updated or snapshot version, is the RLHF model (but bring your own “system prompt”.)

Re:The Waluigi Effect: an explanation of bizarre semiotic ef... by jefftk on Monday March 06, 2023 @08:09PM (#35050181)
Good point! I wonder whether text-davinci-003 is enough to test this?