News for hackers. Stuff that matters.
Re:Deception abilities emerged in large language models by smusamashah on Tuesday June 04, 2024 @06:27PM (#40579609)
At this rate, we will have a paper about every single psychological aspect discovered in LLMs. This could have been just a reddit post.

Every phenomenon found massively in training set will eventually pop up in LLMs. I just don't find the discoveries made in these papers very meaningful.

Edit: May be I am being too short sighted. The researchers probably start from "Humans are good at X and the training data had many examples of X. How good is LLM at X?" and X happens to be deception this time.

Re:Deception abilities emerged in large language models by gwern on Tuesday June 04, 2024 @07:34PM (#40580004)
> Every phenomenon found massively in training set will eventually pop up in LLMs.

That's a nontrivial and controversial claim that many on HN would hotly disagree with - that LLMs are guaranteed to do everything humans do, if you just feed them enough ordinary data, and so finding behaviors like deception or manipulation is not even worth writing up.

And I think even you would abandon that nonchalant position if the "phenomenon" were sufficiently extreme... ("Sure, the LLM autonomously stalked and hunted down a person that a malicious user badmouthed, and called in a swatting using the Twilio API to get the person killed by the police; but you can find plenty of stuff like that in the Internet training corpus, so I don't understand why anyone is interested. This could have just been a reddit post.")

Re:Deception abilities emerged in large language models by picometer on Tuesday June 04, 2024 @07:15PM (#40579905)
I do think it’s an interesting line of inquiry… but not robust enough.

E.g. this paper would be much more interesting if it measured the threshold at which the LLM starts to become good at X, and linked that threshold to the number and character of training examples of X. Then, maybe, we can begin to think about comparing the LLM to a human.

Alas, it requires access to the training data to do that study, and it requires a vast amount of compute to do it robustly.

Re:Deception abilities emerged in large language models by SkyMarshal on Tuesday June 04, 2024 @06:59PM (#40579805)
Even so they should probably get documented in the scientific literature anyway, to encourage review and replication, reduce unintentionally duplicated work, and provide references for further experimentation.
Re:Deception abilities emerged in large language models by picometer on Tuesday June 04, 2024 @07:06PM (#40579848)
Yes, but in a major journal like PNAS? (Proceedings of the National Academy of Sciences)
Re:Deception abilities emerged in large language models by wg0 on Tuesday June 04, 2024 @07:25PM (#40579959)
At this point - is this even real science?

I mean for every single claim made in such papers, these claims are hard to reproduce even with the same model for a set of different inputs.

LLMs have no idea they're adopting some behavior. Just numbers being multiplied and token probabilities being calculated based on the distributions extracted from huge sample corpus.

Re:Deception abilities emerged in large language models by llamaimperative on Tuesday June 04, 2024 @07:37PM (#40580019)
> LLMs have no idea they're adopting some behavior

I suspect you're right, but we absolutely do not know this. We have no idea when, why, or how consciousness emerges. We have no reason to think there's something inherently special about the atoms in an animal brain that imbues them with the capability to "produce" consciousness and that the atoms on a silicon wafer are inherently incapable of doing the same thing.

> Just numbers being multiplied and token probabilities being calculated based on the distributions extracted from huge sample corpus.

This is not wildly different from how brains appear to work. Sure we get a gigantic, constant, highly heterogeneous multimodal stream of data inputs from our sensory systems and we get to test theories about our world model via our own actions, but of course, AI companies are all trying to produce analogs for each of these for their models too.

Re:Deception abilities emerged in large language models by simple_quest_9 on Tuesday June 04, 2024 @07:46PM (#40580065)
LLMs don't have self-perception, egos, etc. right?
Re:Deception abilities emerged in large language models by llamaimperative on Tuesday June 04, 2024 @07:58PM (#40580139)
1. They definitely have self-perception to the extent necessary to "know they're adopting a behavior" if there is something there to "do" the "knowing." They ingest their own outputs as a normal matter of operation.

2. We don't know if there's something conscious there to do the knowing

3. Ego doesn't seem necessary either to being conscious or for self-perception

I suspect you're right, but we don't understand nearly enough about consciousness to be confident about any of these questions.

Re:Deception abilities emerged in large language models by wg0 on Tuesday June 04, 2024 @08:02PM (#40580153)
We don't know is the main argument.

So... Can I worship an LLM and start my own religion? Because we really don't know if consciousness requires X, Y or Z let alone what a a superior all knowing almighty's consciousness consists of.

To me, LLMs look so superior in some ways and maybe they are the gods now that they have manifested themselves as large blobs of floating point numbers when humanity is mature enough to receive them.

We don't know.

Re:Deception abilities emerged in large language models by zzzeek on Wednesday June 05, 2024 @09:16AM (#40584430)
we have every reason to think there's something special about consciousness since it's the one "thing" in "existence" that we all seem to agree exists, yet is 100% unobservable
Re:Deception abilities emerged in large language models by llamaimperative on Wednesday June 05, 2024 @10:58AM (#40585649)
Well yeah, consciousness is special. That's not what I said though. I said there's no reason to think the atoms in an animal (which appear to support consciousness) are special as compared to the atoms in a silicon wafer.
Re:Deception abilities emerged in large language models by zzzeek on Wednesday June 05, 2024 @08:26PM (#40592150)
perhaps, though I think our understanding of what "consciousness" is low enough that we can't assume "atoms working as logic gates" is really sufficient.
Re:Deception abilities emerged in large language models by llamaimperative on Wednesday June 05, 2024 @10:04PM (#40592690)
I didn’t say we should assume it’s sufficient. I said we shouldn’t assume it’s insufficient.