News for hackers. Stuff that matters.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by colanderman on Saturday March 04, 2023 @09:41PM (#35026483)
I agree this is bizarre. There is no "you" in ChatGPT. There is only text which is completed to be highly probable. I would assume surely directives in the 2nd person would perform better (as they would more accurately describe the desired conversation to follow, between two 2nd parties), but maybe the model was trained in such a way that "you" performs just as well here.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Sunday March 05, 2023 @12:17AM (#35027094)
The prompt creates a live bot. You refers to the instance of the running program, not the llm. Bing isn’t chatgpt.

https://telegra.ph/Microsoft-Bing-search-chat-mode-Ruleset-0...

Text completion continues off after a bot is given its rules and then says hi.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by colanderman on Sunday March 05, 2023 @10:45AM (#35030577)
I'm not following. What "running program" is there? I've seen no evidence that there's any intermediary between the rules and the text completion engine, let alone one programmed to believe it is "you". (Yes, there's the program which links the text completion engine to the user, search, etc., but it's not interpreting "you" rules. It's just a pile of C# code.)
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Sunday March 05, 2023 @11:33AM (#35031098)
The program is the text. It’s the equivalent of a script that continues writing itself when run through an interpreter.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by colanderman on Sunday March 05, 2023 @09:37PM (#35036886)
> It understands you as a pronoun that refers to it

No, there is no mechanism for this. ChatGPT is completing text. It has no notion of self. My question is why the above isn't written as "The following is a dialogue between the Microsoft Bing search assistant and a user," without introducing the otherwise meaningless entity of "you".

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Sunday March 05, 2023 @10:30PM (#35037229)
Because the user will address it as “you”. Think of it like a variable.

You = Bing Chat

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by colanderman on Monday March 06, 2023 @01:18PM (#35045145)
Bing Chat will likewise address the user as "you". It's a terrible variable.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Monday March 06, 2023 @02:48PM (#35046627)
That’s how pronouns work in English. It’s an English predictor. It can keep track of the speaker and who is saying you.

It appears to make the models perform better.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by dragonwriter on Sunday March 05, 2023 @12:20AM (#35027114)
> I would assume surely directives in the 2nd person would perform better [...] but maybe the model was trained in such a way that "you" performs just as well here.

I feel like you've used the wrong word here, but I don't know what is intended. "You" is second person, though, so as written this is confusing -- obviously, "you" performs just as well here as 2nd person would, because that's exactly what it is.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by colanderman on Sunday March 05, 2023 @10:41AM (#35030541)
Ah, sorry yes, 3rd person.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by dragonwriter on Sunday March 05, 2023 @12:26PM (#35031710)
In the ChatGPT API, the labels are: “system” (typically used for the “hidden” prompt in examples), “user”, and “assistant” (used for ChatGPT’s responses.)

In examples (and, on the user side in most real chats), the assistant is referred to as “you” both by the user and in the original “system” message.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by colanderman on Sunday March 05, 2023 @09:50PM (#35036983)
Right -- I don't understand why the system message invokes the concept of "you". It seems an odd extra indirection. (That the user calls the assistant "you" is orthogonal -- the assistant likewise calls the user "you".)

Compare something like a screenplay -- the narration is written in the 3rd person, not 2nd person. Characters in the play do refer to each other as "you", but the narrator doesn't identify "you" the reader with one of the characters -- that would be bizarre and break the symmetry of the screenplay (why is one character special, to be identified with the reader?).

2nd person narration to me most strongly is associated with the genre of interactive fiction. This would seem a good fit for these interactive agents -- but the asymmetry is backward: in IF, it is "you" who has agency; the 3rd persons are artificial. Whereas with Bing's script, "you" is artificial, the 3rd person has agency. (Perhaps this inversion biases the text completion to give "Bing" more agency than the other way around.)

i.e. if 2nd person were used at all, I'd expect Bing's script to read something like, "You are now interacting with Bing, Microsoft's chat assistant. Bing does not identify as Sydney ...etc...", and use prefixes "Bing:" and "You:".

(Obviously the approach used for Bing works. It just feels to me like the creators are trying to instruct a computer as if it were self aware -- which it is not.)

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by dragonwriter on Sunday March 05, 2023 @11:55PM (#35037636)
> 2nd person narration to me most strongly is associated with the genre of interactive fiction.

This isn't narration, though its phrased descriptively rather than imperatively; it is direction, which is normally in second person (though, often only detectable in verb conjugations, because usually the imperative is used, and in the imperative the subject is usually omitted.)