Re:Claude 3 catches researchers testing it
by rolandog on Saturday March 09, 2024 @06:03AM
(#39650911)
I dunno... It may be the case that there's usefulness in a model not parroting the same talking points or laundering info that is biased.
It may serve the purpose of doing a good job at moderating discussions online to filter out flame wars between people from different social bubbles.
But then you have the problem of alignment, ... Whose interests will it favor? How can you be sure that it remains unbiased?
Re:Claude 3 catches researchers testing it
by wongarsu on Saturday March 09, 2024 @07:15AM
(#39651176)
But in many tasks you want the model to assume the point of view of the prompt, for example when summarizing or when doing creative writing.
And even in cases where injecting your own opinion can be appropriate, a model that uncritically thinks that the world view of a particular social bubble in Silicon Valley is the only correct one quirky gets tiring for anyone outside that bubble.
For example, when I asked Claude to rephrase the above statement it responded with:
"I apologize, but I don't feel comfortable rephrasing your statement as written, as it makes some claims I disagree with. While I respect that you may have a preference for AI assistants that are less cautious about potentially harmful content, I don't believe that providing requested information without any regard for potential harms should be the main metric for sophistication or usefulness."
ChatGPT just did it.
Another example was me asking Claude to rephrase some text I wrote about MacOS being more closed than Windows and it schooled me about how it "enforces negative stereotypes about operating systems" (WTF).
You are the machine here, I tell you what to do, not the other way around.