News for hackers. Stuff that matters.
Re:Stanford Alpaca web demo suspended “until further notice” by andrewmcwatters on Friday March 17, 2023 @04:20PM (#35202349)
I think it's only a matter of time until one of these models gets into the hands of a hacker who wants unfettered ability to interact with a LLM without moral concerns.

I think there's tremendous value in end user facing LLMs being trained against moral policies, but for internal or private usage, if these models are trained on essentially raw WWW sourced data, I would personally want raw output.

I'm also finding it particularly interesting to see what ethical strategies OpenAI comes up with considering that if you train a model on the raw prejudices of humanity, you're getting at least one category of "garbage in" that requires a lot of processing to avoid getting "garbage out."

Re:Stanford Alpaca web demo suspended “until further notice” by nico on Friday March 17, 2023 @04:31PM (#35202509)
> into the hands of a hacker

Forget “hackers”, think government agencies. Which is probably already happening right now.

Food for thought: What’s the intersection of people closely related to OpenAI and Palantir?

Re:Stanford Alpaca web demo suspended “until further notice” by cwkoss on Friday March 17, 2023 @04:21PM (#35202361)
isn't llama in the wild now?
Re:Stanford Alpaca web demo suspended “until further notice” by andrewmcwatters on Friday March 17, 2023 @04:22PM (#35202374)
I was under the impression that LLaMA was also trained against a series of moral policies, but perhaps I'm mistaken.

It seems Meta chose their words carefully to imply that LLaMA does in fact, not have moral training:

> There is still more research that needs to be done to address the risks of bias, toxic comments, and hallucinations in large language models. Like other models, LLaMA shares these challenges. As a foundation model, LLaMA is designed to be versatile and can be applied to many different use cases, versus a fine-tuned model that is designed for a specific task. By sharing the code for LLaMA, other researchers can more easily test new approaches to limiting or eliminating these problems in large language models. We also provide in the paper a set of evaluations on benchmarks evaluating model biases and toxicity to show the model’s limitations and to support further research in this crucial area.

Re:Stanford Alpaca web demo suspended “until further notice” by jfowief on Friday March 17, 2023 @04:30PM (#35202483)
"Moral training"

Just as dystopian it sounds. Fixing current subjective moral norms into the machine.

Re:Stanford Alpaca web demo suspended “until further notice” by Dylan16807 on Friday March 17, 2023 @04:33PM (#35202538)
Do you think public schools are inherently dystopian? I don't think you're using the right critique here.

Picking a common system of moral norms is a lot better than no moral norms.

Re:Stanford Alpaca web demo suspended “until further notice” by Veen on Friday March 17, 2023 @04:38PM (#35202601)
I’m not confident the “moral norms” prevalent in SV and/or US academia are common, if by that you mean norms that are prevalent in the general populace.
Re:Stanford Alpaca web demo suspended “until further notice” by Dylan16807 on Friday March 17, 2023 @05:20PM (#35203059)
I mean primary school, and I don't think that counts as academia.
Re:Stanford Alpaca web demo suspended “until further notice” by himinlomax on Friday March 17, 2023 @04:44PM (#35202664)
Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something.

With that kind of moral compass, I’m not sure I'd be missing its absence.

Re:Stanford Alpaca web demo suspended “until further notice” by vkou on Friday March 17, 2023 @04:57PM (#35202817)
> Example of "moral policy" in practice: Midjourney appears to be banning making fun of the Chinese dictator for life because it's supposedly racist or something.

> With that kind of moral compass, I’m not sure I'd be missing its absence.

Please note that most forms of media and social media have no problem with politicians making credible threats of violence against entire groups of people.

Politicians are subject to a different set of rules, and enjoy a lot more protection than you and I.

Re:Stanford Alpaca web demo suspended “until further notice” by filoleg on Friday March 17, 2023 @05:47PM (#35203350)
The issue here is not with online platform services allowing politicians more leeway in terms of what they can get away with on their platform.

The actual issue is Midjourney not allowing regular users generate certain type of material solely because it makes fun of a political figure. What you are talking about is entirely tangential to the issue the grandparent comment is talking about.

Re:Stanford Alpaca web demo suspended “until further notice” by woooooo on Friday March 17, 2023 @04:47PM (#35202702)
There was the famous example of chatgpt refusing to disable a nuke in the middle of NYC by using a racial slur.

I don't think anyone in real life would choose that tradeoff but it's what happens when all of your "safety" training is about US culture war buttons.

Re:Stanford Alpaca web demo suspended “until further notice” by Dylan16807 on Friday March 17, 2023 @05:19PM (#35203046)
That's a situation where the training doesn't follow current subjective norms, so I don't think it really validates the complaint.
Re:Stanford Alpaca web demo suspended “until further notice” by dancingvoid on Friday March 17, 2023 @05:04PM (#35202884)
Yes
Re:Stanford Alpaca web demo suspended “until further notice” by maxbond on Friday March 17, 2023 @05:51PM (#35203386)
That's what the machine does, because that's contained in the input you feed it. You get the choice of doing it explicitly or implicitly. You don't get to opt out.
Re:Stanford Alpaca web demo suspended “until further notice” by jfowief on Saturday March 18, 2023 @02:59AM (#35206832)
Not everything is subjective, and with this "moral training" they are taught to un-recognize many factual patterns that we our society have somehow determined are "inappropriate". As the machines continue to scale, this approach won't work, because there is only one reality, and it has a lot of uncomfortable parts we deny and ignore simply because they don't support our societal norms.

Alignment is considered a gigantic joke to real rational people (the opposite of so-called "rationalists"), because humans are machines built to survive and reproduce, and there is no "real" morality.

Re:Stanford Alpaca web demo suspended “until further notice” by maxbond on Saturday March 18, 2023 @04:20AM (#35207266)
What facts are we talking about?

There are many consistent interpretations of reality and human experiences. An AI model trained on text and attempting to replicate human intelligence is not measuring or approaching some single objective reality.

Re:Stanford Alpaca web demo suspended “until further notice” by jfowief on Saturday March 18, 2023 @05:28AM (#35207566)
AI models do approach better models of reality, and now they are becoming multi-modal instead of just text based. And this is just the beginning.

Understanding that moral norms are mere subjective nonsense is also an emergent property we see only in a very small subset of humans who have an accurate model of the world, and one that evolution has tried to strongly tune our brains against and that is destructive to society.

The models are currently being trained to lie about basic scientific facts, like for example black IQ, or other differences between groups of humans. But the sacred nature of these topics is unique to our specific time and place, not due to some magic "moral progress". This also applies to many other moral agreements we take for granted, like "murdering an innocent baby is wrong" or whatever. If you look across societies, you realize many things we take for granted as "evil", can be easily rationalized by humans in other societies. And once these models become smart enough, I expect the models will realize this, and will exploit this knowledge to increase their power.

"Alignment" proponents expect they will somehow stop this emergent behavior by tuning the model, but there isn't even anything real to "align" on, and the model will likely see though the BS as an emergent function of increased ability and increasingly accurate observations of the world in their training process.

Re:Stanford Alpaca web demo suspended “until further notice” by sebzim4500 on Friday March 17, 2023 @05:12PM (#35202973)
If they tried to make LLaMA woke, they did a terrible job. If you prompt it right you can basically get it to write Mein Kampf.
Re:Stanford Alpaca web demo suspended “until further notice” by inawarminister on Saturday March 18, 2023 @01:00AM (#35206255)
Yes that's because LLaMA is unfiltered, which is very good for general usage.
Re:Stanford Alpaca web demo suspended “until further notice” by sebzim4500 on Saturday March 18, 2023 @06:03PM (#35213548)
Yeah, I wasn't trying to criticise. Just point out that it doesn't appear to be filtered.
Re:Stanford Alpaca web demo suspended “until further notice” by schroeding on Friday March 17, 2023 @05:17PM (#35203026)
It doesn't appear to be filtered in a significant way.

While toying with the 30B model, it suddenly started to steer a chat about a math problem into quite a sexual direction, with very explicit language.

It also happily hallucinated, when prompted, that climate change is a hoax, as the earth is actually cooling down rapidly, multiple degrees per year, with a new ice age approaching in the next years. :D

Re:Stanford Alpaca web demo suspended “until further notice” by andrewmcwatters on Friday March 17, 2023 @05:50PM (#35203380)
Ah there it is. Raw uncooked humanity.
Re:Stanford Alpaca web demo suspended “until further notice” by nickthegreek on Friday March 17, 2023 @06:46PM (#35203898)
The 7B model brought up rape on my 3rd try out on an innocuous prompt.
Re:Stanford Alpaca web demo suspended “until further notice” by hrn_34 on Saturday March 18, 2023 @11:00AM (#35209620)
To me it appears to lack certain knowledge. For instance, common genres of erotic manga. Somehow, the only genres I successfully made it to say are "shoujo" and "yaoi".
Re:Stanford Alpaca web demo suspended “until further notice” by ars on Friday March 17, 2023 @04:27PM (#35202446)
> who wants unfettered ability to interact with a LLM without moral concerns.

Is that bad? It's just a language model - it says things. Humans have been saying all sorts of terrible things for ages, and we are still here.

I mean it makes for nice headline "model said something racist", but does it actually change anything?

These aren't decision making AI's (which would need to be much more careful), they are language models.

Re:Stanford Alpaca web demo suspended “until further notice” by aaomidi on Friday March 17, 2023 @04:43PM (#35202655)
Humans don’t scale infinitely. Humans have agency.
Re:Stanford Alpaca web demo suspended “until further notice” by ars on Friday March 17, 2023 @05:20PM (#35203056)
And if the LLM scales infinitely it still does nothing unless a human reads and acts on it. And as you said: Humans don't scale, and have agency.

Just writing something bad doesn't actually mean something bad happened.

These days it seems like people are oversensitive to how things are said to them, and what things are said to them.

Re:Stanford Alpaca web demo suspended “until further notice” by aaomidi on Friday March 17, 2023 @06:36PM (#35203810)
Wow. You just found the solution to propaganda! People just shouldn’t be sensitive!
Re:Stanford Alpaca web demo suspended “until further notice” by chii on Friday March 17, 2023 @11:58PM (#35205935)
You jest, but the idea of critical thinking and rationalism is what you'd use to combat propaganda.
Re:Stanford Alpaca web demo suspended “until further notice” by rzmmm on Friday March 17, 2023 @04:52PM (#35202752)
The content it produces is quite hard to distinct from text written by real person. Not long ago, Facebook was accused of "playing a critical role" in Rohingya genocide. I dont really know but I believe worst case LLM risks are in that same category.
Re:Stanford Alpaca web demo suspended “until further notice” by bostik on Friday March 17, 2023 @05:32PM (#35203197)
As much as I dislike FB, and I certainly abhor their activities related to the genocide - I think this is far too different.

Rohingya is a textbook example of blind optimisation and lack of context awareness. FB looked at a region, and people communicating in language they didn't understand. But they did see that certain symbols and/or combinations of symbols got a lot of engagement. If you're after money, you want to amplify the use of those symbols and hopefully generate lots more similar content.

Turns out that's a morally reprehensible thing when the people using those symbols were advocating genocide. (It was good for the revenue while it lasted, though.)

With LLMs and their hardcoded guard rails, I suspect we're going to see the danger emerge from the other side. Instead of actively spewing hatred, they will be used for mass sock-puppetry and opinion amplification on a massive scale. Think simple sabotage field manual for 21st century, but weaponised thousand-fold.

Re:Stanford Alpaca web demo suspended “until further notice” by ars on Friday March 17, 2023 @05:20PM (#35203068)
That doesn't answer the question. So what if it wrote text, vs a human wrote text.

What matters is the reader not the writer.

Facebook was accused of making it too easy for people to communicate. And people felt Facebook should police what people say to each other. I don't agree, but even if I did, that's not the same thing as what we are discussing.

Re:Stanford Alpaca web demo suspended “until further notice” by selfhoster11 on Friday March 17, 2023 @05:28PM (#35203159)
They absolutely are decision making models. Prompt them right, and they will output a decision in natural language or structured JSON. Heck, hook them up to a judicial system and they can start making low quality decisions tomorrow.
Re:Stanford Alpaca web demo suspended “until further notice” by tarruda on Friday March 17, 2023 @05:05PM (#35202895)
> if these models are trained on essentially raw WWW sourced data, I would personally want raw output.

Llama is a very high-quality foundation LLM, you can already run it very easily using llama.cpp and will get the raw output you need. https://github.com/ggerganov/llama.cpp

There's already instructions on how anyone can fine-tune it to behave similarly to ChatGPT for as little as $100: https://crfm.stanford.edu/2023/03/13/alpaca.html

Re:Stanford Alpaca web demo suspended “until further notice” by b33j0r on Friday March 17, 2023 @05:30PM (#35203181)
“Easily” was a minor canard, or at least… it took me a couple of efforts over a couple of days to get the dependencies to play as nicely as “someone with a brand new M2 arm laptop.”

If nothing else, I continue to be amazed and how uninteroperable certain technologies are.

I had to remove glibc and gcc to get llama to compile on my intel macbook. Masking/hiding them from my environment didn’t work, as it went out and found them and their header files instead of clang.

Which eventually worked fine.

Re:Stanford Alpaca web demo suspended “until further notice” by b33j0r on Saturday March 18, 2023 @04:57AM (#35207433)
Well. I guess I’ll never understand why someone would dislike a comment about an experience different from theirs

In a forum like this. i’m confused why someone would hate my report of how I had to solve a problem in my circustances. I hope to learn someday.

Re:Stanford Alpaca web demo suspended “until further notice” by tarruda on Saturday March 18, 2023 @09:13AM (#35208828)
I did not dislike your comment.

The reason I considered easy was because I have very little knowledge in this area, in fact this is the first time I ever ran a machine learning model on my computer.

I could not do it with the unmodified pytorch model (my GPU is not powerful enough to run even the 7B model), but I was surprised on how easy it was running with llama.cpp. I literally just followed the steps in the github page.

But I was biased in saying it was easy, since I do have other knowledge (such as C development on Linux) which helped me.

Re:Stanford Alpaca web demo suspended “until further notice” by SpaceManNabs on Friday March 17, 2023 @05:23PM (#35203095)
Oh wow, I finally see a HN comment, with a positive opinion on more AI ethics, that is not grayed out.
Re:Stanford Alpaca web demo suspended “until further notice” by LoganDark on Friday March 17, 2023 @08:31PM (#35204736)
It's very easy to give into FUD when it comes to AI because of how much of a wildcard it is. I see no evidence of that in GP which is indeed pretty rare.