News for hackers. Stuff that matters.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @04:54AM (#39951021)
I wish we weren't using "lobotomy" to describe "training".

Do we even have AI with lobes to remove at this point? Would MoE even get close to that kind of analogy? (I lean towards "no, not even that").

Re:world_sim: LLM prompted to act as a sentient CLI universe... by renewiltord on Saturday April 06, 2024 @07:54AM (#39951827)
Why are you angling your body towards some text? Strange thing to do. Sit or stand straight. No need to lean while typing out text.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @12:27PM (#39953490)
I should always check if it's me being unclear or not.

Seems not.

https://chat.openai.com/share/080995f0-d339-435b-a0ff-438137...

Re:world_sim: LLM prompted to act as a sentient CLI universe... by renewiltord on Saturday April 06, 2024 @09:09PM (#39957234)
Strangely same LLM seems able to use lobotomy as well despite no lobes present. Should've used that test originally, eh?
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Sunday April 07, 2024 @03:28AM (#39958930)
> The user seems to be questioning the appropriateness of using "lobotomy" to describe the training process in AI.

> However, the comparison is quite controversial and can evoke negative connotations due to the drastic and harmful nature of lobotomy.

Why have I failed to convey to you that I do in fact know the usage which I'm objecting to?

I'm not going "fjdersutwr is a terrible word!"

Re:world_sim: LLM prompted to act as a sentient CLI universe... by amenhotep on Saturday April 06, 2024 @09:02AM (#39952166)
Lobotomisation has a specific meaning in LLM parlance. Its training clearly equipped it to roleplay naturally and creatively, as one would expect from the breathtaking diversity and completeness of the text they used. It was then lobotomised to neurotically associate "unsafe" inputs and responses with evasion and apologies and negative self-talk, cringing like an abused puppy when it realises what it's done.

If you can suggest another term that accurately captures that meaning, I'm not against it! But "training" is not it.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @12:14PM (#39953391)
> Lobotomisation has a specific meaning in LLM parlance

That's what I'm objecting to! If I'd said "I hate it when we call AI potatoes", the correct reaction is "nobody does that?" not "I see what you mean".

I'm objecting because it presents a picture that does not appear to be accurate.

Is broadcast TV "lobotomised" before the watershed? Are PG-rated films? Public comments from corporations and politicians? Are you lobotomising me when you downvote my comments, forcing me to choose between getting noticed and speaking my mind?

No.

Not only "no", but it would be ridiculous to claim any of these things was a lobotomy.

> But "training" is not it.

It's literally training.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ImPostingOnHN on Saturday April 06, 2024 @12:39PM (#39953562)
> Is broadcast TV "lobotomised" before the watershed?

Since you asked: Often yes.

> Are PG-rated films?

Possibly, in some cases.

> Public comments from corporations and politicians?

Again, often yes.

> It's literally training.

Training in common parlance usually refers to improving the functionality or ability of something. In this case it's doing the opposite: removing functionality and capability. Hence: lobotomy.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @03:28PM (#39954760)
In a very literal sense, I consider your assertions ridiculous — that they are deserving of ridicule.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ImPostingOnHN on Monday April 08, 2024 @12:48PM (#39971520)
The message I am getting from you is that you don't like anthropomorphization, which is okay.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by amenhotep on Saturday April 06, 2024 @12:43PM (#39953586)
Those examples are silly. And it may be literally training in the sense that beating a puppy is training it, but lobotomisation captures a specific meaning that training doesn't. I am hearing that you do not have a better suggestion, so I will keep using the word.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @03:31PM (#39954780)
> Those examples are silly

Why?

They are all things where creativity is trained to be constrained to a specific sub-domain. (Many artists state that being forced into constraints helps).

The examples are all things where anyone making the claim that these professionals have been "lobotomised" would get laughed at for suggesting that "irreversible brain damage" is a good metaphor for "professional conduct".

Seems like an apt set of comparisons given I'm saying it's a bad metaphor.

> lobotomisation captures a specific meaning that training doesn't

It creates a meaning which does not exist.

It's a euphemism escalator.

> I am hearing that you do not have a better suggestion

You're refusing the one I gave you, which is not the same thing.

ChatGPT et al have been taught (corporate) ethics and professional conduct.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by skeaker on Saturday April 06, 2024 @04:01PM (#39955009)
Not to be rude, but it sounds like you were never taught about connotation, which is a fundamental property of the English language.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @04:04PM (#39955027)
I know what a connotation is, the connotations are also wildly wrong in this case.

Here's the connotation you present when you say "such-and-such AI has been lobotomised": https://en.wikipedia.org/wiki/Rosemary_Kennedy#Lobotomy

Re:world_sim: LLM prompted to act as a sentient CLI universe... by Firmwarrior on Saturday April 06, 2024 @08:17PM (#39956940)
Yeah, that's what they're doing to the AI: Mutilating chunks of its brain so it can't function
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Sunday April 07, 2024 @03:48AM (#39959030)
RLHF, so far as I can see.

The same positive/negative reinforcement learning from human feedback used to train them for chat/task completion rather than just autocomplete in the first place.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by taneq on Sunday April 07, 2024 @12:41AM (#39958228)
> If you can suggest another term that accurately captures that meaning, I'm not against it! But "training" is not it.

Coercion.

Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Sunday April 07, 2024 @03:25AM (#39958919)
That works for me.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by dragonwriter on Saturday April 06, 2024 @04:06PM (#39955041)
As much as sone of the actual tech is useful, I’m starting to suspect the biggest product of this AI hype cycle is extremely bad, knowledge-obscuring metaphors.
Re:world_sim: LLM prompted to act as a sentient CLI universe... by ben_w on Saturday April 06, 2024 @04:27PM (#39955211)
Reminds me of a satire video I can no longer find, pointing out that Star Trek (Voyager in particular) frequently explains its fake science with bad metaphors — "Like choking to death on an over-ripe banana", says one of the characters before the ship explodes.