News for hackers. Stuff that matters.
Re:What We Know About LLMs (A Primer) by jannyfer on Tuesday July 25, 2023 @09:03AM (#36861586)
Great summary.

I’ve been reading a pop neuroscience book called Incognito (2011).

In it, the author talks about how the brain is a group of competing sub-brains of many forms, and the brain might have several ways of doing the same thing (e.g. recognizing an object). The author also posited that the lack of AI progress back then was due to the fact that there are no constantly competing sub-brains. Our brains are always adjusting and trying new scenarios.

I was struck by how similar these brain observations were to recent developments in AI and LLMs.

The book is full of cool stories, even if some of them are now recognized as non-reproducible. I recommend!

Re:What We Know About LLMs (A Primer) by go_elmo on Tuesday July 25, 2023 @09:12AM (#36861696)
In the end - an AI should have these competing subsystems in one system - just as our brains are one system. What I find extremely interesting is how perception and thinking differs from person to person too - it was a "taboo" topic to call this neurodiversity - just as other genetic traits, but AI makes this relevant more than ever imo. Sure, its complicated and much comes from nurture (Nurture vs nature.. as exposure/epigeneticd vs genetics) but there sure are markable differences - the ones starting to stand out are e.g. adhd / autistic people, but Im sure it wont stay just there over time!
Re:What We Know About LLMs (A Primer) by samwillis on Tuesday July 25, 2023 @09:24AM (#36861863)
You touch in an important topic here, how our understanding of AI/ML/LLMs will influence our "understanding" of the human brain and intelligence.

My fear is that we will ascribe too much human behaviour to that we see in and understand of our AI inventions, and that this will result in the dehumanisation of people.

So essentially my fear is what we justify doing to each other due to AI, rather than what "AGI" could do to us.

Re:What We Know About LLMs (A Primer) by lacrimacida on Friday July 28, 2023 @05:54PM (#36913883)
I started seeing this dehumanisation spring up even here on HN, comparing LLMs with human brains and human thoughts as similar regurgitation to LLMs. I’m afraid it will get worse as this technology advances
Re:What We Know About LLMs (A Primer) by simonh on Tuesday July 25, 2023 @10:17AM (#36862600)
Even within my immediate family we seem to have distinct differences in our conscious experience. My wife has very little visual or auditory experience of thought, no inner voice even when reading a book. While I mostly experience speaking as a continuous stream of words coming basically from my subconscious, with only a vague sense of what's coming up, one of my daughters says she is consciously aware of the exact words she is going to say several seconds in advance. It's like she has the ability to introspect her internal speech buffer, while I can't.

So while I'm sure there are a lot of custom tuned, problem specific hardware structures in our brain architecture, we do seem to learn how to actually use that hardware individually. As a result we seem to come up with a diverse range of different high level approaches.

Re:What We Know About LLMs (A Primer) by throwuwu on Tuesday July 25, 2023 @09:21AM (#36861809)
How are they similar?
Re:What We Know About LLMs (A Primer) by esafak on Tuesday July 25, 2023 @03:34PM (#36868118)
> The author also posited that the lack of AI progress back then was due to the fact that there are no constantly competing sub-brains.

That became popular in neural networks after the introduction of dropout regularization, which forced neurons to "co-adapt" and learn to do each others' jobs. Large, over-specified models also provide a natural setting for co-adaptation.

Re:What We Know About LLMs (A Primer) by moffkalast on Tuesday July 25, 2023 @07:08PM (#36870983)
Isn't dropout just there to avoid overfitting? This is more like a mixture of experts type architecture.
Re:What We Know About LLMs (A Primer) by esafak on Tuesday July 25, 2023 @08:22PM (#36871542)
That is one lens to view it through. Co-adaptation reduction is another, and it is an intuitive one: generalization ability is improved if a neuron has to be predictive in multiple contexts, and can't rely on other neurons to lift all the weight, if you pardon the pun.

Improving neural networks by preventing co-adaptation of feature detectors https://arxiv.org/abs/1207.0580

Re:What We Know About LLMs (A Primer) by masswerk on Tuesday July 25, 2023 @08:08PM (#36871440)
In fact, this is what psychoanalysis and the notion of the unconscious (as opposed to "subconscious processes") was all about. (And it's also, where the "talking cure" found its leverage.)
Re:What We Know About LLMs (A Primer) by wilhelm____ on Wednesday July 26, 2023 @08:35AM (#36876605)
thank you!