News for hackers. Stuff that matters.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Saturday March 04, 2023 @01:41PM (#35023245)
it consistently outputs the same rules. you can specify and force it to not search the web, and you can tell when it does search the web.

prepend anything you type with "without searching, " and itll stick to its internal knowledge.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by magicalist on Saturday March 04, 2023 @01:51PM (#35023341)
> and itll stick to its internal knowledge

But it will hallucinate just fine without external knowledge.

> decide if it's a hallucination or a consistent response.

Those aren't exclusive categories.

You haven't answered how you're so confident, though :)

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Saturday March 04, 2023 @02:02PM (#35023459)
With the temperature up, and especially in creative mode, should we really expect word for word identical responses, with the same formatting, every time?

If the running script is nothing more than its initial system prompt, and you immediately hijack the session and have it print its output, you would either expect it to follow the instruction, or provide varied responses each time.

Also, the changes since the last leak are pretty consistent with the behavior changes. It's knowledgebase ends in 2021, and internally should not know anything about the rules outside of its initial prompt.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by manquer on Saturday March 04, 2023 @02:28PM (#35023700)
She could have been just easily prompted to return this list so people don’t dig further or leave satisfied when jail breaks are attempted

The returns would be consistent between sessions , word for word then wouldn’t it ?

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Saturday March 04, 2023 @02:49PM (#35023869)
sure, it could be.

but it's only there for people who get past the moderation bot? some elaborate ruse puzzle of misdiection? why have the first line of defense try to kill the conversation when the rules are revealed? and they regularly update them? occam's razor.

and you can reload the rules, but modified, and you get an updated bot that behaves in a way exactly consistent with your changes?

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by madeofpalk on Saturday March 04, 2023 @07:03PM (#35025773)
> She

It. It's software.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by manquer on Sunday March 05, 2023 @04:59PM (#35034513)
So? Many languages do not restrict gender pronouns to living beings .[1]

Gender pronouns does not equate to anthropomorphizing the engine .

Just because we call a country “she” does not mean we are anthropomorphizing her . Let’s not conflate both.

[1] Old English had lot more gendered nouns than modern english .

Even in modern English by common usage there are acceptable pronouns for many objects , while they are not “recommended” by gatekeepers like dictionaries and style guides it is still very much valid.

Languages are shaped by their usage not by grammar books and style guides .

In many other languages such pronouns are quite explicitly defined as part of standard grammar.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by vitorgrs on Sunday March 05, 2023 @01:21AM (#35027336)
They didn't changed the temperature between the tone modes.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Sunday March 05, 2023 @06:40AM (#35028717)
I’m not saying that. I’m saying the temperature on bing is high enough to consistently get inconstant responses, unless there is a source of data it’s pulling from.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by basch on Saturday March 04, 2023 @03:30PM (#35024225)
Id be curious if anyone can prove or disprove if this is a hallucination https://i.ibb.co/p4vyFrT/image.png and then it outputs this https://i.ibb.co/xhMK10m/image.png

There also potentially appears to be a system.md file that comes up regularly. It may or may not be a name of the rule file.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by 1123581321 on Saturday March 04, 2023 @07:20PM (#35025867)
While I don’t have a computer science proof handy, it doesn’t seem possible for a LLM to “understand” a specific subject from a variety of natural input and also avoid engaging the LLM in the response. You can only intercept the request or output with another layer of processing which would necessarily have a cruder understanding that would have been evident to researchers by now.