News for hackers. Stuff that matters.
Re:ChatGPT Has Gone Berserk by caymanjim on Wednesday February 21, 2024 @02:08AM (#39450948)
Is this meant to be how the ChatGPT designers/operators instruct ChatGPT to operate? I guess I shouldn't be surprised if that's the case, but I still find it pretty wild that they would parameterize it by speaking to it so plainly. They even say "please".
Re:ChatGPT Has Gone Berserk by tarruda on Wednesday February 21, 2024 @02:43AM (#39451115)
> I still find it pretty wild that they would parameterize it by speaking to it so plainly

Not my area of expertise, but they probably fine tuned it so that it can be parametrized this way.

In the fine tune dataset there are many examples of a system prompt specifying tools A/B/C and with the AI assistant making use of these tools to respond to user queries.

Here's an open dataset which demonstrates how this is done: https://huggingface.co/datasets/togethercomputer/glaive-func.... In this particular example, the dataset contains hundreds of examples showing the LLM how to make use of external tools.

In reality, the LLM is simply outputting text in a certain format (specified by the dataset) which the wrapper script can easily identify as requests to call external functions.

Re:ChatGPT Has Gone Berserk by Grimblewald on Wednesday February 21, 2024 @02:43AM (#39451114)
If you want to go the stochastic parrot route (which i dont fully biy) then because statistically speaking a request paired with please is more likely to be met, then the same is true for requests passed to a LLM. They really do tend to respond better when you use your manners.
Re:ChatGPT Has Gone Berserk by EchoChamberMan on Wednesday February 21, 2024 @05:01PM (#39460240)
It is a stochastic parrot, and you perfectly explain why saying please helps.
Re:ChatGPT Has Gone Berserk by bowsamic on Wednesday February 21, 2024 @02:43AM (#39451118)
That's how prompt injection usually works, isn't it?
Re:ChatGPT Has Gone Berserk by herbst on Wednesday February 21, 2024 @05:15AM (#39452060)
From my experience with 3.5 I can confirm that saying please or reasoning really helps to get whatever results you want. Especially if you want to manifest 'rules'
Re:ChatGPT Has Gone Berserk by Arn_Thor on Wednesday February 21, 2024 @04:41PM (#39459964)
There's a certain logic to it, if I'm understanding how it works correctly. The training data is real interactions online. People tend to be more helpful when they're asked politely. It's no stretch that the model would act similarly.