News for hackers. Stuff that matters.
Re:Do Large Language Models learn world models or just surfa... by ramraj07 on Sunday January 22, 2023 @01:52AM (#34475058)
Isn’t our brain a black box too? Why exactly is it worrisome?
Re:Do Large Language Models learn world models or just surfa... by tobr on Sunday January 22, 2023 @02:10AM (#34475133)
Because human brains have been known to intentionally lie and manipulate, and secretly plan horrible deeds.
Re:Do Large Language Models learn world models or just surfa... by eternalban on Sunday January 22, 2023 @02:29PM (#34481351)
We'll need psych tests for our AI. Malicious AI will note this and adapt. An arms races ensues, where one adversary finances and invests in upgrading the other. Guess who wins?
Re:Do Large Language Models learn world models or just surfa... by shadowofneptune on Sunday January 22, 2023 @02:13AM (#34475143)
The mind itself, perhaps. The person in possession of it though usually does have a conscious strategy to play a game once they are proficient, and can happily explain it to you.
Re:Do Large Language Models learn world models or just surfa... by pyinstallwoes on Sunday January 22, 2023 @02:15AM (#34475149)
How many people can actually explain their state of mind and rational or seemingly rational thought process?
Re:Do Large Language Models learn world models or just surfa... by space_fountain on Sunday January 22, 2023 @02:24AM (#34475187)
And we have all sorts of social science these days implying the explanations when people do try to explain themselves they’re justifying decisions that came out of black box rather than actual describing how the decision happened
Re:Do Large Language Models learn world models or just surfa... by nequo on Sunday January 22, 2023 @02:51AM (#34475318)
And also neuroscience examining whether what we think we did out of choice was really done out of our choice:

https://en.wikipedia.org/wiki/Neuroscience_of_free_will

Re:Do Large Language Models learn world models or just surfa... by taneq on Sunday January 22, 2023 @08:15AM (#34477045)
What do you (or they) think “our choice” is if not the result of these processes? It’s not like “you” are just the past of your brain that is consciously aware of making decisions.
Re:Do Large Language Models learn world models or just surfa... by Tepix on Sunday January 22, 2023 @04:21AM (#34475706)
I play Othello. In most situations i can give you detailed reasons why i choose to make a particular move.
Re:Do Large Language Models learn world models or just surfa... by taneq on Sunday January 22, 2023 @08:46AM (#34477301)
Are they actually the reasons for your move, or are they post-hoc justifications, and how would you know?
Re:Do Large Language Models learn world models or just surfa... by Isinlor on Sunday January 22, 2023 @12:34PM (#34479882)
You are not explaining your internal processes (neurobiology, brain chemistry, neuron firing patterns etc.) that caused you to arrive at given moves. You will only provide proofs that your moves are good.
Re:Do Large Language Models learn world models or just surfa... by bobbruno on Sunday January 22, 2023 @04:47PM (#34482569)
You can justify post facto the move you made. Can you write down the logical process (probably a combination of strategy, move generation, analysis and selection) that would replicate the moves you picked from the beginning of the game? I don’t even need the brain chemistry (the implementation), but I don’t think even writing down the logic that generates those moves (in contrast with a logic to justify each move after it was generated) is something humans can do in most cases.
Re:Do Large Language Models learn world models or just surfa... by bsenftner on Sunday January 22, 2023 @07:51AM (#34476881)
Asking people to explain themselves often triggers various sorts of youth punishment memory, resulting in them being both uncomfortable probing their own reasoning, and then they produce what is essentially a comforting deception for their reasoning simple for the sake of ending the agony of self probing. Asking people their internal states requires their internal guard and internal (potentially filled with self deception) voice what they are up to is very similar to asking children why they are misbehaving, and all manners of mental protections activate.
Re:Do Large Language Models learn world models or just surfa... by gpm on Sunday January 22, 2023 @04:47AM (#34475829)
I strongly disagree with this statement. A good player can usually walk you through the portion of the game tree that they expanded in their head, but the moves they had to think about (expand their mental game tree to include the resulting position) versus the moves they instinctively ruled out is basically inexplicable, and where a lot of the skill lies.

(I'm thinking more chess than othello, since I'm much better at the former, but I assume the thought process is similar)

Re:Do Large Language Models learn world models or just surfa... by ch33zer on Sunday January 22, 2023 @05:25AM (#34476012)
Strongly agree. I'm NOT good at chess, but I still like watching it on YouTube. The explanations that top players give for their moves sound like magic a lot of the time. If they sat down with me for an hour could they explain every step of how they got to their conclusion? I think they could, but I don't think that they'd explain the exact same process they actually went through, which is based on inexplicable heuristic as parent mentioned. Often times at complex decision points in chess players will just say that a move feels better. That feeling can't be explained.
Re:Do Large Language Models learn world models or just surfa... by shadowofneptune on Sunday January 22, 2023 @05:57AM (#34476172)
I have to admit you're right there, now thinking about how I would explain games I play, approaches to tasks, etc.
Re:Do Large Language Models learn world models or just surfa... by jacquesm on Sunday January 22, 2023 @09:02AM (#34477479)
That's not correct. It doesn't matter whether they can explain their moves to you. What matters is if they can explain their moves to another grand master and they always can unless they moved a random piece to break up a position to make a boring game more interesting (this rarely happens, but it is an interesting way to play, but not to win contests).
Re:Do Large Language Models learn world models or just surfa... by Isinlor on Sunday January 22, 2023 @12:32PM (#34479853)
They are not explaining their internal processes (neurobiology, brain chemistry, neuron firing patterns etc.) that caused them to arrive at given moves. They just provide proofs that their moves were good.
Re:Do Large Language Models learn world models or just surfa... by jacquesm on Sunday January 22, 2023 @03:10PM (#34481707)
Works for me. Just like you won't be able to explain to a lay person what a computer program does doesn't mean that the logic is sound without going all the way to the atomic level. But if you really wanted to you could because a computer is a deterministic and relatively simple system, small enough to explain in principle from which we can extrapolate. Wetware doesn't have that luxury so you have the choice to believe the explainer, or not. But when dealing with such a vast gap in experience the best way is to try to learn as much as possible from such an explanation in the hope that you'll be able to capture more of it in the future. This isn't always possible, for instance due to time constraints or interest levels but in principle it could be done.
Re:Do Large Language Models learn world models or just surfa... by squokko on Sunday January 22, 2023 @02:50AM (#34475306)
A person with a million loyal adherents is worrisome even if those adherents are only sitting on computers. For similar reasons, a person with a trillion unflinchingly loyal AI adherents is worrisome.
Re:Do Large Language Models learn world models or just surfa... by xwolfi on Sunday January 22, 2023 @03:04AM (#34475371)
What, why? Unless they vote or are convincing to humans, it's not that worrisome. I mean end of the day, I care about eating well, raising my kids in peace, being useful to others: if an AI tries to convince me to act, it'd have to factor these constraints, in which case it's a useful fellow human able to understand my need and proposing an action for debate.

Wars and disasters are usually pursued by minority groups in charge of means of violence, and manipulation is a way for them to trap you into letting a crack open long enough for them to invade it, but if we cant resist an AI, they deserve the Earth more than us :)

Re:Do Large Language Models learn world models or just surfa... by stackbutterflow on Sunday January 22, 2023 @04:03AM (#34475625)
Because the world has had thousand years to evolve, adapt and settle around the capabilities and limitations of the human brain. It is somewhat stable and predictable.

Now if a new species of humans suddenly emerged with a brain not only has capable as our but faster, more adaptable and extensible, it would disrupt the world.

Re:Do Large Language Models learn world models or just surfa... by belugacat on Sunday January 22, 2023 @07:10AM (#34476581)
Ecosystems (what I infer you mean by "the world", since things that are part of the world such as gravity, plate tectonics, etc don't follow the laws of evolution) haven't had thousands of years to evolve, adapt, and settle around the capabilities that we've had since the industrial revolution started. AI is merely an aspect and consequence of the logical progression of that continuity.

(In terms of tangible effect on the world today and what seems on track for the few decades to come, AI is still far behind coal - to pick one - when it comes to concrete negative externalities tho)

Re:Do Large Language Models learn world models or just surfa... by gjm11 on Sunday January 22, 2023 @07:23AM (#34476672)
I think something is wrong with the logic of your first paragraph.

Suppose (by way of analogy; this is not meant to be directly addressing AI) that in the next year some ingenious physicists finally get quantum gravity figured out, and that soon after the papers are published someone notices that the theory implies that one can use common household materials to make a device capable of destroying a planet.

(Seems unlikely, but I don't think anyone would have predicted that the last revolutionary theory of gravity we worked out would imply that one can use not-so-common materials to make a reasonably small device capable of destroying a city, but it turns out it does.)

It seems fairly likely that (1) the knowledge of how to do this could not be suppressed for ever, and that (2) if making the device turns out not to need all the fancy apparatus and hard-to-get materials that e.g. hydrogen bombs require, it wouldn't be that long before some idiot actually does it and destroys the earth.

And yet, these new discoveries would be "merely an aspect and consequence of the logical progression of that continuity", as you put it.

So, whatever the actual situation is with AI, I think it is demonstrably not the case that we can be confident AI won't somehow kill us all or destroy the world merely because AI is a thing we made and the world has had thousands of years to adapt to us.

Maybe everything will be fine, maybe not, but figuring out which will require looking in more detail rather than handwaving about how ecosystems always adapt.

Re:Do Large Language Models learn world models or just surfa... by the__prestige on Sunday January 22, 2023 @10:26AM (#34478230)
So both humans and AI could theoretically destroy the planet if allowed to continue to evolve. Heck, ants might do that too. Does that establish anything?
Re:Do Large Language Models learn world models or just surfa... by gjm11 on Sunday January 22, 2023 @04:47PM (#34482570)
Nope. It means that if you are concerned about the possibility of something destroying the planet (or whatever), you actually have to look at what it is and what it can do and so forth.
Re:Do Large Language Models learn world models or just surfa... by Retric on Sunday January 22, 2023 @07:30AM (#34476721)
Ecosystems are experiencing the 6th (soon to be 5th) largest mass extinction due to those capabilities. https://en.wikipedia.org/wiki/Holocene_extinction
Re:Do Large Language Models learn world models or just surfa... by jacquesm on Sunday January 22, 2023 @09:00AM (#34477456)
It isn't and it could well be much, much worse depending on some of the initial conditions, plenty of which we haven't really charted to the point that we can infer outcomes from inputs. It's chaos theory all over but this time with an unpredictable black box in the middle.
Re:Do Large Language Models learn world models or just surfa... by mihaic on Sunday January 22, 2023 @05:30AM (#34476028)
We are forced to work with the imperfections of the human mind, and a great deal of effort in our world is put into working with them.

Why would we want more of this?

Re:Do Large Language Models learn world models or just surfa... by taylorius on Sunday January 22, 2023 @07:46AM (#34476840)
You make it sound like the human brain is a shoddy piece of work. In any case, even if it is imperfect, it isn't imperfect because it is a black box.
Re:Do Large Language Models learn world models or just surfa... by mihaic on Tuesday January 24, 2023 @01:46PM (#34508087)
We sometimes make life-changing decisions based on where Jupiter was when we were born.

I mean, we've been struggling for milenia against our biases. I'd let some time pass until we can decide on what these ML models can get subtly wrong.

Re:Do Large Language Models learn world models or just surfa... by bigbluedots on Sunday January 22, 2023 @06:34AM (#34476391)
So that if it starts killing people at some point, you can debug it or maybe figure out why it has done that.
Re:Do Large Language Models learn world models or just surfa... by meindnoch on Sunday January 22, 2023 @09:57AM (#34477942)
We're all driving one, so at least we can extrapolate from our own experience.
Re:Do Large Language Models learn world models or just surfa... by dweinus on Sunday January 22, 2023 @01:18PM (#34480407)
Because, as a human, I can introspect my world model. I can explain it to myself and others. I can explain how it guides my decisions. I can directly update it without needing "training data". I don't have a map of my neurons, but I have self awareness. For LLMs, there is the opposite: transparent structure but opaque understanding.
Re:Do Large Language Models learn world models or just surfa... by d1sxeyes on Monday January 23, 2023 @03:08AM (#34486567)
Computers have historically been 'correct' about things. Obviously, humans who program computers make mistakes, and occasionally, things like cosmic bit flips can occur, leading a computer to produce an incorrect answer.

An LLM upends that paradigm, and worse, the closer it comes to appearing to reason and discuss, the more it implies that the human brain is nothing more than a biological computer, which is worrying for a lot of people.

If humans are nothing more than biological computers, do (or can) silicon computers have souls? If a collection of diodes can have a soul, what makes us special?