French AI startup Mistral releases ‘unmoderated’ chatbot via torrent

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by simonw on Saturday September 30, 2023 @05:28PM (#37720014)

These examples come up a lot. It's important to note that this isn't actually fully representative of the base Llama 2 model - it's greatly exaggerated by the Llama 2 default system prompt, which is this:

    You are a helpful, respectful and honest
    assistant. Always answer as helpfully as possible, while
    being safe.  Your answers should not include any
    harmful, unethical, racist, sexist, toxic, dangerous, or
    illegal content. Please ensure that your responses are
    socially unbiased and positive in nature.

    If a question does not make any sense, or is not
    factually coherent, explain why instead of answering
    something not correct. If you don't know the answer to a
    question, please don't share false information.

If you change that system prompt when you run Llama and it will be much less likely to tell you not to kill a Linux process or whatever.

[ Parent | on HN ]

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by crooked-v on Saturday September 30, 2023 @07:49PM (#37721100)

I think it's telling, though, that however they've trained it results in stuff like equating even referencing the existence of the song "Power" with being 'harmful' and 'unethical' enough to lecture the user about it.

[ Parent | on HN ]

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by refulgentis on Saturday September 30, 2023 @07:57PM (#37721151)

no, they haven't. those terms are in the system prompt. llama2 is a base model, llama2 chat has some training but not a ton. that's why you're seeing big gains from further rlhf on it.

[ Parent | on HN ]

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by crooked-v on Saturday September 30, 2023 @07:59PM (#37721161)

The point you're missing is that the actual associations between those terms and other concepts are baked into the model, such that 'harmful'<->'Kanye West music' is a strong enough association for it to actively refuse to answer the question once prompted that way.

[ Parent | on HN ]

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by refulgentis on Saturday September 30, 2023 @10:54PM (#37722107)

no it's not

there is a sizable portion of people who genuinely believe things like this so here we are. just sort of sniping at eachother ineffectually. i can't prove it to you without someone owning up to whats in the training data

[ Parent | on HN ]

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by salad-tycoon on Sunday October 01, 2023 @03:56AM (#37723760)

I don’t know if there actually is a sizable group of people or a sizable group of influencers who astroturfed enough public awareness into a sizable group of people.

That output is just a highly concentrated sample of what academic and cultural influencers have been paid to promote either on TikTok or Reddit or the classroom.

A problem that will only increase as the ability to put on mass propaganda campaigns virtually only gets cheaper and more effective with the help your friendly AI assistant.

No that output is trite rightspeak gobblygook that no one could possibly believe without significant coaching on tv, social media, the classroom, and finally your HR mandatory education requirements.

addendum: every now and then I’ll notice a mass influencing campaign because I will get a PING from Apple News about a story, typically about topics and from news sources that Ive explicitly banned. Shortly afterwards PING! not just the same story but same catch words and sentence structure starts popping up everywhere else on the internet. So yeah I think some of what is on the internet is forced programming made to look like a majority opinion and AI is being trained on that not actual people’s opinions.

[ Parent | on HN ]

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by Topgamer7 on Saturday September 30, 2023 @08:21PM (#37721281)

I'll look around for using MLC, but it's effectively useless for me at the moment. It literally won't even write a loving hallmark card. Trust me, I tried.

[ Parent | on HN ]

Re:French AI startup Mistral releases ‘unmoderated’ chatbot ... by whywhywhywhy on Sunday October 01, 2023 @06:07PM (#37731382)

Interesting, thanks for pointing this out. Definitely was all running on defaults and the chat model when I tried.

[ Parent | on HN ]