News for hackers. Stuff that matters.
Re:Cargo Cult AI by version_five on Saturday August 05, 2023 @01:22PM (#37014289)
This is just wrong, it has no external goals, it just predicts next tokens or behaves in some other way that has minimized a training loss. It doesn't matter what you "plug it in to", it will just do what you tell it. You could speculate there might be instructions that lead to emergent behavior, but then your back to just speculating about how AI might work. Current llms don't work the way you're implying.
Re:Cargo Cult AI by ryanjshaw on Saturday August 05, 2023 @01:40PM (#37014530)
> it has no external goals

Where do you believe humans get their "external goals" from?

> It doesn't matter what you "plug it in to", it will just do what you tell it.

Here's a ChatGPT-4 transcript where I told the LLM it's controlling a human harness: https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512...

Other than my initial instructions (which all humans receive from other humans!), where did it "do what I told it"?

Re:Cargo Cult AI by almost_usual on Saturday August 05, 2023 @01:50PM (#37014639)
You only perform tasks instructed to you by other people?
Re:Cargo Cult AI by version_five on Saturday August 05, 2023 @01:54PM (#37014683)
There's some philosophical question here obviously. We could be the emergent behavior of our atoms desire ot oxidize things. But I don't belive that has any testability or value as an argument when discussing whether computer programs, especially NNs predicting next tokens can become intelligent. At best the argument could be "we don't know what intelligence is so maybe it's that" which holds no water.
Re:Cargo Cult AI by almost_usual on Saturday August 05, 2023 @02:01PM (#37014739)
Do NN discover new tokens or encounter spontaneous tokens on its own?
Re:Cargo Cult AI by pixl97 on Saturday August 05, 2023 @03:48PM (#37015697)
Please give a good definition of 'on their own' and what that entails.

And conversely to the spontaneousness of current AI, your body has a constant set of inputs from reality. That is you never stop feeling, hearing, seeing, sensing, etc. Your brain can consciously turn lower the sensitivity on these things (sleeping). Now, if we subject a multimodal AI this continuous stream, how will it behave?

AI is currently compute and power limited. Very little research has gone into continuous powerhungry AI that goes off and does its own thing at this point. And I would counter that it might be really dumb to design such a device without understanding the risks it entails.

Re:Cargo Cult AI by ryanjshaw on Saturday August 05, 2023 @02:05PM (#37014789)
Did you make an honest attempt to think through the question?

You are the product of very long line of humans vs. environment, nature & nurture, cultural values, etc. Do you believe the way you generate your next set of "tokens" (thoughts, actions) is completely independent of your "training" as a human? Is your response to a given stimulus completely random?

Re:Cargo Cult AI by almost_usual on Saturday August 05, 2023 @02:13PM (#37014853)
Can an LLM discover novel tokens on its own?
Re:Cargo Cult AI by natch on Saturday August 05, 2023 @03:23PM (#37015455)
You'd have to define LLM and "on its own."

Can the LLM have a runloop? Can the LLM be situated in a world like you and me are?

If the LLM is just a file on a hard disk in a drawer not connected to anything, then obviously it can't discover novel tokens on its own.

If on the other hand the LLM has a runloop and sensors and basic instructions to do observations and run thought experiments and find new combinations of concepts and name them with tokens, then sure, why wouldn't it be able to?

You might say you define LLMs as "LLMs as they exist today in a human prompt-driven system" but that would be an artificial limitation given the trivial level of programming, even simple bash scripting, that would be necessary to give an LLM a runloop, access to sensors, and basic instructions to discover new stuff.

Re:Cargo Cult AI by oefnak on Saturday August 05, 2023 @03:23PM (#37015458)
Can all humans?
Re:Cargo Cult AI by IanCal on Saturday August 05, 2023 @03:32PM (#37015542)
How many people invent new words or letters?
Re:Cargo Cult AI by kalkin on Saturday August 05, 2023 @05:05PM (#37016287)
Can you make a novel sound? One that's not part of any human language?

Perhaps you can, using a tool. However, if we're allowing tools, I bet GPT4 could also write a program that would produce a novel token, by whatever definition you might give.

I don't think GPT4 is AGI. But this is not a good test. (And it does mean something that coming up with a good test is increasingly nontrivial.)

Re:Cargo Cult AI by andrepd on Saturday August 05, 2023 @02:15PM (#37014876)
Bruh, the LLM has parsed the entirety of Zork, plus maybe thousands of articles written (by humans) on it. At least pick a better example.
Re:Cargo Cult AI by ryanjshaw on Saturday August 05, 2023 @02:33PM (#37015042)
Bro, you want me to come up with an example that doesn't exist anywhere in the OpenAI training data? They've probably trained it on every single piece of fiction and non-fiction that exists!

I would have to come up with something no human has ever documented. I don't think that is possible, or what point it would make, since nobody would be able to assess the quality of the output in that context?

Re:Cargo Cult AI by burnished on Saturday August 05, 2023 @02:51PM (#37015191)
Yes, come up with a novel example. An original story is still possible.
Re:Cargo Cult AI by pixl97 on Saturday August 05, 2023 @03:55PM (#37015744)
> An original story is still possible

Is it? The names might possible be original, and maybe the exact flow of ideas, but it's insanely rare for someone to come up with a new concept and not an amalgamation of existing ideas.

Re:Cargo Cult AI by goaaron on Saturday August 05, 2023 @03:08PM (#37015315)
It is very easy to come up with something novel. Unless you don’t interact with the world.
Re:Cargo Cult AI by YeGoblynQueenne on Saturday August 05, 2023 @06:50PM (#37017006)
I don't understand the point of this experiment. You ask ChatGPT to generate some text, and it generates some text. Rather, that's what it's programmed to do and it generates text following from your prompt. What does your transcript demonstrate?

I also have to point out that even if you could build a ... human harness? (I'm not sure what that is exactly, but I'm sort of guessing) it would be a little mad to expect that ChatGPT could control it simply by saying what it does.

The ability to generate text when prompted is not enough to make an agent capable of autonomously interacting with the world.

Re:Cargo Cult AI by Philadelphia on Saturday August 05, 2023 @01:31PM (#37014401)
It also can’t learn. Once the training is done, the network is set in stone.
Re:Cargo Cult AI by danielbln on Saturday August 05, 2023 @01:33PM (#37014437)
Technically it can do in-context learning (and really well, too), but that's not persisted into the network.
Re:Cargo Cult AI by kenjackson on Saturday August 05, 2023 @02:03PM (#37014764)
And that just seems like an engineering problem. Not something that is considered intractable.
Re:Cargo Cult AI by version_five on Saturday August 05, 2023 @03:39PM (#37015607)
It's easy to say that, but "surely it must be possible to connect an llm in such a way that it becomes intelligent" (tell me if I'm misinterpreting) is not a demonstration of anything. It's basically restating the view from the 50s that with computers having been invented, an intelligent computer is a short way off.
Re:Cargo Cult AI by dinvlad on Saturday August 05, 2023 @02:15PM (#37014880)
Learning != inference. Learning is development of generalizable knowledge, of which this is simply not, since it can’t extend beyond the short context window.
Re:Cargo Cult AI by adamisom on Saturday August 05, 2023 @01:33PM (#37014427)
Makes me wonder why we don’t see deployed models that keep learning during inference.
Re:Cargo Cult AI by LegitShady on Saturday August 05, 2023 @02:39PM (#37015102)
Microsoft tay has entered the chat
Re:Cargo Cult AI by Der_Einzige on Saturday August 05, 2023 @04:37PM (#37016073)
The curse of dimensionality and exploding/vanishing gradients are why incremental learning is still so rare.
Re:Cargo Cult AI by optimalsolver on Sunday August 06, 2023 @06:28AM (#37020618)
Re:Cargo Cult AI by ryanjshaw on Saturday August 05, 2023 @01:43PM (#37014560)
What do you mean by "learn"?

The network has learned human patterns of language, knowledge and information processing. If you want to update that, you can re-train it on a regular basis, and re-play its sensory/action history to "restore" its state.

If you mean "learn from experience", (1) a lot of that is pointless because it's already learned from the experiences of millions of humans through their writing and (2) LLMs can "learn" when you explain consequences.

Re:Cargo Cult AI by pixl97 on Saturday August 05, 2023 @03:57PM (#37015763)
In theory they could learn by having their discussions fed back to them in the future, and it does seem that this occurs.

Now, there is no continuous learning in the human/animal sense. Of course it is thought that even humans have to sleep and re-weight their networks so short term knowledge is converted to long term knowledge.