News for hackers. Stuff that matters.
Re:Predict the next frame. Results of training a transformer... by MorbidCuriosity on Tuesday December 05, 2023 @08:49PM (#38539560)
I think the last hurdles will be sentience and a theory of the mind. Theory of the mind is probably something that a LM could predict given enough data, but sentience? I don't think we understand enough about our own sentience for us to create it in a machine.
Re:Predict the next frame. Results of training a transformer... by naasking on Wednesday December 06, 2023 @12:22AM (#38540720)
> I don't think we understand enough about our own sentience for us to create it in a machine.

Invention preceding understanding is the norm, not the exception. We created fire before understanding chemistry, and we constantly use pharmaceuticals without really understanding how they work. Invention first, then theory comes along to explain and generalize.

For all we know, sentience is a necessary side-effect of semantic processing of any kind, in which case LLMs already have a form of sentience.

So yes, you're right that we don't understand our own sentience. In fact, we understand so little that it could literally be staring us in the face right now and we don't realize it.

Re:Predict the next frame. Results of training a transformer... by kenmorechalfant on Tuesday December 05, 2023 @11:05PM (#38540372)
It goes back to the old question of whether matter is fundamental and consciousness emerges from it or the other way around. My inclination is the latter, by way of Descartes. "I think therefore I am." If you can think "I am me" then you know with certainty that your first person identity exists but all other information that comes to you through your senses will always be an incomplete picture of the material universe. I think this is the strongest argument why consciousness precedes matter (or pervades through all matter, depending how you look at it). If sentience is a fundamental property of the universe, the way I see it, all matter shares the same soul. That makes it perfectly logical for a machine that mimics the pathways of the brain to behave just like a brain, to me.
Re:Predict the next frame. Results of training a transformer... by og_kalu on Tuesday December 05, 2023 @10:52PM (#38540315)
LLMs can already predict theory of mind in text and keep good track of character motivations and knowledge in a story.
Re:Predict the next frame. Results of training a transformer... by jononor on Wednesday December 06, 2023 @05:51AM (#38542363)
It may be that we found out that sentience is not as magical or unique as we originally thought.

Some people claimed that "creativity" (another rather nebulous term) was a uniquely human trait, and machines could perform tasks that require this. But recent generative AI models have started to make people question that position.

Re:Predict the next frame. Results of training a transformer... by fisf on Wednesday December 06, 2023 @07:12AM (#38542818)
Neither of those are required for intelligence.