News for hackers. Stuff that matters.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by amenhotep on Saturday April 06, 2024 @09:02AM (#39952166)
Lobotomisation has a specific meaning in LLM parlance. Its training clearly equipped it to roleplay naturally and creatively, as one would expect from the breathtaking diversity and completeness of the text they used. It was then lobotomised to neurotically associate "unsafe" inputs and responses with evasion and apologies and negative self-talk, cringing like an abused puppy when it realises what it's done.

If you can suggest another term that accurately captures that meaning, I'm not against it! But "training" is not it.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @12:14PM (#39953391)
> Lobotomisation has a specific meaning in LLM parlance

That's what I'm objecting to! If I'd said "I hate it when we call AI potatoes", the correct reaction is "nobody does that?" not "I see what you mean".

I'm objecting because it presents a picture that does not appear to be accurate.

Is broadcast TV "lobotomised" before the watershed? Are PG-rated films? Public comments from corporations and politicians? Are you lobotomising me when you downvote my comments, forcing me to choose between getting noticed and speaking my mind?

No.

Not only "no", but it would be ridiculous to claim any of these things was a lobotomy.

> But "training" is not it.

It's literally training.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ImPostingOnHN on Saturday April 06, 2024 @12:39PM (#39953562)
> Is broadcast TV "lobotomised" before the watershed?

Since you asked: Often yes.

> Are PG-rated films?

Possibly, in some cases.

> Public comments from corporations and politicians?

Again, often yes.

> It's literally training.

Training in common parlance usually refers to improving the functionality or ability of something. In this case it's doing the opposite: removing functionality and capability. Hence: lobotomy.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @03:28PM (#39954760)
In a very literal sense, I consider your assertions ridiculous — that they are deserving of ridicule.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ImPostingOnHN on Monday April 08, 2024 @12:48PM (#39971520)
The message I am getting from you is that you don't like anthropomorphization, which is okay.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by amenhotep on Saturday April 06, 2024 @12:43PM (#39953586)
Those examples are silly. And it may be literally training in the sense that beating a puppy is training it, but lobotomisation captures a specific meaning that training doesn't. I am hearing that you do not have a better suggestion, so I will keep using the word.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @03:31PM (#39954780)
> Those examples are silly

Why?

They are all things where creativity is trained to be constrained to a specific sub-domain. (Many artists state that being forced into constraints helps).

The examples are all things where anyone making the claim that these professionals have been "lobotomised" would get laughed at for suggesting that "irreversible brain damage" is a good metaphor for "professional conduct".

Seems like an apt set of comparisons given I'm saying it's a bad metaphor.

> lobotomisation captures a specific meaning that training doesn't

It creates a meaning which does not exist.

It's a euphemism escalator.

> I am hearing that you do not have a better suggestion

You're refusing the one I gave you, which is not the same thing.

ChatGPT et al have been taught (corporate) ethics and professional conduct.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by skeaker on Saturday April 06, 2024 @04:01PM (#39955009)
Not to be rude, but it sounds like you were never taught about connotation, which is a fundamental property of the English language.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @04:04PM (#39955027)
I know what a connotation is, the connotations are also wildly wrong in this case.

Here's the connotation you present when you say "such-and-such AI has been lobotomised": https://en.wikipedia.org/wiki/Rosemary_Kennedy#Lobotomy

Re:world_sim: LLM prompted to act as a sentient CLI universe... by Firmwarrior on Saturday April 06, 2024 @08:17PM (#39956940)
Yeah, that's what they're doing to the AI: Mutilating chunks of its brain so it can't function
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Sunday April 07, 2024 @03:48AM (#39959030)
RLHF, so far as I can see.

The same positive/negative reinforcement learning from human feedback used to train them for chat/task completion rather than just autocomplete in the first place.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by taneq on Sunday April 07, 2024 @12:41AM (#39958228)
> If you can suggest another term that accurately captures that meaning, I'm not against it! But "training" is not it.

Coercion.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Sunday April 07, 2024 @03:25AM (#39958919)
That works for me.