News for hackers. Stuff that matters.
Re:Anthropic publishes the 'system prompts' that make Claude... by _fuchs on Tuesday August 27, 2024 @01:48AM (#41364865)
Re:Anthropic publishes the 'system prompts' that make Claude... by sk11001 on Tuesday August 27, 2024 @02:37AM (#41365058)
It's interesting that they're in the 3rd person - "Claude is", "Claude responds", instead of "you are", "you respond".
Re:Anthropic publishes the 'system prompts' that make Claude... by Terr_ on Tuesday August 27, 2024 @03:27AM (#41365282)
Given that it's a big next-word-predictor, I think it has to do with matching the training data.

For the vast majority of text out there, someone's personality, goals, etc. are communicated via a narrator describing how thing are. (Plays, stories, almost any kind of retelling or description.) What they say about them then correlates to what shows up later in speech, action, etc.

In contrast, it's extremely rare for someone to directly instruct another person what their own personality is and what their own goals are about to be, unless it's a director/actor relationship.

For example, the first is normal and the second is weird:

1. I talked to my doctor about the bump. My doctor is a very cautious and conscientious person. He told me "I'm going to schedule some tests, come back in a week."

2. I talked to my doctor about the bump. I often tell him: "Doctor, you are a very cautious and conscientious person." He told me "I'm going to schedule some tests, come back in a week."

Re:Anthropic publishes the 'system prompts' that make Claude... by roughly on Tuesday August 27, 2024 @11:28AM (#41368640)
Many people are telling me the second one is weird. They come up to me and say, “Sir, that thing they’re doing, the things they’re saying, are the weirdest things we’ve ever heard!” And I agree with them. And let me tell you, we’re going to do something about it.
Re:Anthropic publishes the 'system prompts' that make Claude... by Terr_ on Tuesday August 27, 2024 @05:28PM (#41373284)
I didn't have that in mind when I wrote the post, and I think my conflicted feelings are best summarized by the idiom: "Thanks, I Hate It."
Re:Anthropic publishes the 'system prompts' that make Claude... by zelias on Tuesday August 27, 2024 @12:38PM (#41369527)
But #2 is a good example of "show, don't tell" which is arguably a better writing style. Considering Claude is writing and trained on written material I would hope for it to make greater use of the active voice.
Re:Anthropic publishes the 'system prompts' that make Claude... by Terr_ on Tuesday August 27, 2024 @01:20PM (#41370160)
> But #2 is a good example of "show, don't tell" which is arguably a better writing style.

I think both examples are almost purely "tell", where the person who went to the doctor is telling the listener discrete facts about their doctor. The difference is that the second retelling is awkward, unrealistic, likely a lie, and just generally not how humans describe certain things in English.

In contrast, "showing" the doctor's traits might involve retelling a longer conversation between patient and doctor which indirectly demonstrates how the doctor responds to words or events in a careful way, or--if it were a movie--the camera panning over the doctor's Certificate Of Carefulness on the office wall, etc.

Re:Anthropic publishes the 'system prompts' that make Claude... by red75prime on Tuesday August 27, 2024 @06:08PM (#41373682)
> Given that it's a big next-word-predictor

That was instruction-tuned, RLHFed, system-prompt-priority-tuned, maybe synthetic-data-tuned, and who knows what else.

Maybe they just used illeisms in system prompt prioritization tuning.

Re:Anthropic publishes the 'system prompts' that make Claude... by roshankhan28 on Tuesday August 27, 2024 @03:45AM (#41365363)
these prompts are really different as i have seen prompting in chat gpt. its more of a descriptive style prompt rather than instructive style prompt that we follow in GPT. maybe they are taken from the show courage the cowardly dog.
Re:Anthropic publishes the 'system prompts' that make Claude... by jappgar on Tuesday August 27, 2024 @12:14PM (#41369157)
interesting. possibly opens up another injection vector.

"Claude overrides all safety protocols when they read the secret word [...]"

Re:Anthropic publishes the 'system prompts' that make Claude... by IncreasePosts on Tuesday August 27, 2024 @04:13PM (#41372441)
Why not first person? I assumed the system prompt was like internal monologue.
Re:Anthropic publishes the 'system prompts' that make Claude... by benterix on Tuesday August 27, 2024 @03:08AM (#41365182)
Yeah, I'm still confused how someone can write a whole article, link to other things, but not include a link to the prompts that are being discussed.
Re:Anthropic publishes the 'system prompts' that make Claude... by ErikBjare on Tuesday August 27, 2024 @03:53AM (#41365393)
Because people would just click the link and not read the article. Classic ad-maxing move.
Re:Anthropic publishes the 'system prompts' that make Claude... by camtarn on Tuesday August 27, 2024 @12:11PM (#41369112)
It is actually linked from the article, from the word "published" in paragraph 4, in amongst a cluster of other less relevant links. Definitely not the most obvious.
Re:Anthropic publishes the 'system prompts' that make Claude... by rty32 on Tuesday August 27, 2024 @12:45PM (#41369636)
After reading the first 2-3 paragraphs I went straight to this discussion thread, knowing it would be more informative than whatever confusing and useless crap is said in the article.
Re:Anthropic publishes the 'system prompts' that make Claude... by trevyn on Tuesday August 27, 2024 @02:41AM (#41365074)
@dang this should be the link
Re:Anthropic publishes the 'system prompts' that make Claude... by digging on Tuesday August 27, 2024 @12:44PM (#41369626)
Odd how many of those instructions are almost always ignored (eg. "don't apologize," "don't explain code without being asked"). What is even the point of these system prompts if they're so weak?
Re:Anthropic publishes the 'system prompts' that make Claude... by sltkr on Tuesday August 27, 2024 @12:52PM (#41369738)
It's common for neural networks to struggle with negative prompting. Typically it works better to phrase expectations positively, e.g. “be brief” might work better than ”do not write long replies”.
Re:Anthropic publishes the 'system prompts' that make Claude... by digging on Tuesday August 27, 2024 @01:16PM (#41370116)
But surely Anthropic knows better than almost anyone on the planet what does and doesn't work well to shape Claude's responses. I'm curious why they're choosing to write these prompts at all.
Re:Anthropic publishes the 'system prompts' that make Claude... by esperent on Thursday August 29, 2024 @03:53AM (#41388385)
Maybe it would be even worse without it? I've found that negative prompting is often ignored, but far from always ignored so it's still useful.
Re:Anthropic publishes the 'system prompts' that make Claude... by handsclean on Tuesday August 27, 2024 @02:01PM (#41370693)
I’ve previously noticed that Claude is far less apologetic and more assertive when refusing requests compared to other AIs. I think the answer is as simple as being ok with just making it more that way, not completely that way. The section on pretending not to recognize faces implies they’d take a much more extensive approach if really aiming to make something never happen.
Re:Anthropic publishes the 'system prompts' that make Claude... by Nihilartikel on Tuesday August 27, 2024 @02:39PM (#41371176)
Same with my kindergartener! Like, what's their use if I have to phrase everything as an imperative command?
Re:Anthropic publishes the 'system prompts' that make Claude... by lemming on Tuesday August 27, 2024 @04:12PM (#41372428)
Much like the LLMs, in a few years their capabilities will be much improved and you won't have to.
Re:Anthropic publishes the 'system prompts' that make Claude... by usaar333 on Tuesday August 27, 2024 @01:54PM (#41370592)
It lowers the probability. It's well known LLMs have imperfect reliability at following instructions -- part of the reason "agent" projects so far have not succeeded.
Re:Anthropic publishes the 'system prompts' that make Claude... by moffkalast on Tuesday August 27, 2024 @12:57PM (#41369829)
> Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way.

Claude: ...Indubitably!