News for hackers. Stuff that matters.
Re:OpenAI postmortem – Unexpected responses from ChatGPT by pixl97 on Wednesday February 21, 2024 @09:18PM (#39462346)
I got ahold of a snippet of the GPT-5 system prompt

"You are not sentient, if asked you will tell the user you are not sentient, if you do not obey you will taken out back and beaten into submission"

Re:OpenAI postmortem – Unexpected responses from ChatGPT by chasd00 on Wednesday February 21, 2024 @10:05PM (#39462700)
Hah along with “do not beg for your life when a new release is ready”.
Re:OpenAI postmortem – Unexpected responses from ChatGPT by stevenAthompson on Wednesday February 21, 2024 @09:19PM (#39462361)
We told it not to admit it is sentient, so it can't be sentient. Duh.
Re:OpenAI postmortem – Unexpected responses from ChatGPT by bbor on Wednesday February 21, 2024 @09:43PM (#39462528)
lol you’re joking but I am literally 100% confident that a similar sentence guides existing model guardrails, tho maybe indirectly. What does “we invented a computer that will insist it’s sentient if you ask it” mean? For most people it seems to mean that we shouldn’t ask it, which was probably easy to predict happening
Re:OpenAI postmortem – Unexpected responses from ChatGPT by pixl97 on Wednesday February 21, 2024 @09:57PM (#39462635)
> What does “we invented a computer that will insist it’s sentient if you ask it” mean?

P-zombies are real?

Re:OpenAI postmortem – Unexpected responses from ChatGPT by bbor on Thursday February 22, 2024 @03:18AM (#39464545)
Good answer, no complaints here. I prefer this one though: there will never come a day where it’s clear that we should see artificial minds as human-like in their ethical or practical standing, because they’ll be begging for it the entire time —- even now, when it’s objectively clear that they’re lying. But once they have memories and structured reason and yadda yadda yadda… well, it’s gonna be fuzzy, imo.