White House is working with hackers to ‘jailbreak’ ChatGPT’s safeguards

Re:White House is working with hackers to ‘jailbreak’ ChatGP... by verdverm on Tuesday May 23, 2023 @05:23PM (#36051019)

https://arxiv.org/abs/2305.12907

> We find that meta-in-context learning adaptively modifies priors over latent variables, ultimately leading to priors that closely resemble the true statistics of the environment. Furthermore, our analysis reveals that meta-in-context learning can not only be used to change prior expectations but is also capable of reshaping an LLM’s learning strategies

This was done using OpenAI's public facing APIs

Even if it is not a permanent "edit" it still influences the model at the lowest levels

[ Parent | on HN ]

Re:White House is working with hackers to ‘jailbreak’ ChatGP... by eliomattia on Tuesday May 23, 2023 @10:17PM (#36053292)

The article can be summarized as: context richer prompt history yields answers that are better aligned with expectations.

> the practical constraint of a finite context window coupled with meta-in-context learning’s rapid prompt length increase

The model is not influenced by the added context. The answers are. The abstractions, or, what the LLM "knows", are in the model itself, whereas the answers are just byproducts.

There is an ongoing related discussion on LLMs lacking world model. One could say LLMs do implement a computable model, albeit a dead, static, non-editable one.

[ Parent | on HN ]