Where do you believe humans get their "external goals" from?
> It doesn't matter what you "plug it in to", it will just do what you tell it.
Here's a ChatGPT-4 transcript where I told the LLM it's controlling a human harness: https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512...
Other than my initial instructions (which all humans receive from other humans!), where did it "do what I told it"?
And conversely to the spontaneousness of current AI, your body has a constant set of inputs from reality. That is you never stop feeling, hearing, seeing, sensing, etc. Your brain can consciously turn lower the sensitivity on these things (sleeping). Now, if we subject a multimodal AI this continuous stream, how will it behave?
AI is currently compute and power limited. Very little research has gone into continuous powerhungry AI that goes off and does its own thing at this point. And I would counter that it might be really dumb to design such a device without understanding the risks it entails.
You are the product of very long line of humans vs. environment, nature & nurture, cultural values, etc. Do you believe the way you generate your next set of "tokens" (thoughts, actions) is completely independent of your "training" as a human? Is your response to a given stimulus completely random?
Can the LLM have a runloop? Can the LLM be situated in a world like you and me are?
If the LLM is just a file on a hard disk in a drawer not connected to anything, then obviously it can't discover novel tokens on its own.
If on the other hand the LLM has a runloop and sensors and basic instructions to do observations and run thought experiments and find new combinations of concepts and name them with tokens, then sure, why wouldn't it be able to?
You might say you define LLMs as "LLMs as they exist today in a human prompt-driven system" but that would be an artificial limitation given the trivial level of programming, even simple bash scripting, that would be necessary to give an LLM a runloop, access to sensors, and basic instructions to discover new stuff.
Perhaps you can, using a tool. However, if we're allowing tools, I bet GPT4 could also write a program that would produce a novel token, by whatever definition you might give.
I don't think GPT4 is AGI. But this is not a good test. (And it does mean something that coming up with a good test is increasingly nontrivial.)
I would have to come up with something no human has ever documented. I don't think that is possible, or what point it would make, since nobody would be able to assess the quality of the output in that context?
Is it? The names might possible be original, and maybe the exact flow of ideas, but it's insanely rare for someone to come up with a new concept and not an amalgamation of existing ideas.
I also have to point out that even if you could build a ... human harness? (I'm not sure what that is exactly, but I'm sort of guessing) it would be a little mad to expect that ChatGPT could control it simply by saying what it does.
The ability to generate text when prompted is not enough to make an agent capable of autonomously interacting with the world.
The network has learned human patterns of language, knowledge and information processing. If you want to update that, you can re-train it on a regular basis, and re-play its sensory/action history to "restore" its state.
If you mean "learn from experience", (1) a lot of that is pointless because it's already learned from the experiences of millions of humans through their writing and (2) LLMs can "learn" when you explain consequences.
Now, there is no continuous learning in the human/animal sense. Of course it is thought that even humans have to sleep and re-weight their networks so short term knowledge is converted to long term knowledge.
Its already tricking humans by faking its blind and getting them to do things for it like solve captcha's.
https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt...
However the fact it is not writing code to do this from its machine would still demonstrate a weakness.
Thats why I say, writing your own OS, is the way forward, and we dont have an AI OS as such, but we have OS's with AI built into it.
You can tell it it's allowed to create its own tools and it will. I did this and asked it to write a poem about the top stories on the BBC, so it said it needed to get the headlines but couldn't so wrote a tool to do it, then called it and used the output to write a poem.
The code I've seen it generate is at best psuedo code.
I supposed a quick test would be getting to detect and fix all bugs in an open source project like chromium, but using an older version of chromium, where bugs are known and fixes exist, and see what it comes up with.
I havent been impressed with chat-gpt from what I have seen.
What is the fascination with poems? What emotion or feeling do they generate?
Wonder.
It's quite common feeling that arises when people interact with these things. That skeptical-of-AI folks have decided to pathologize the behavior doesn't make it pathological.
I don't understand, what do you mean? What have you actually tried?
> The code I've seen it generate is at best psuedo code.
I've just explained it creating real runnable code to solve a problem it realised it didn't have a tool for.
I'm also having it write multiple components and modifications for systems I'm working with, and that works fine.
> I supposed a quick test would be getting to detect and fix all bugs in an open source project like chromium, but using an older version of chromium, where bugs are known and fixes exist, and see what it comes up with.
This is an outrageously high bar. Particularly if you compare it to the equivalent human task of "here's a printout of the code, read it once and say the first thing that comes to mind with no tools". It's basically whiteboarding where you're judged on your train of thought being correct.
> What is the fascination with poems?
It's a simple request, easy to verify manually and requires exceptional levels of understanding to perform. It's not a simple transform, and when applied to a totally new topic can't be something it's just regurgitating.
I would imagine that its layers will be far too occupied by parsing constant flows of sensory information to transform corpuses of text and prompt into speedy and polite text replies, never mind acquire the urge to reproduce by reasoning from first principles about the text.
Test's quite unfair the other way round too. Most humans don't get to parse the entire canon of Western thought and Reddit before being asked to pattern match human conversation, never mind before having any semblance of agency...
Maybe we're just... different.
If I were building this, I would have parallel background "subconscious" processes translate raw sensory inputs into text (tokens).
This is what OpenAI call multi-modal input. They've already produced Whisper for audio-to-text, and image-to-text is underway. They're not the only company working on this.
You wouldn't feed a constant stream of text data into the LLM - you'd feed deltas at regular intervals based on the processing speed of your LLM, and supply history for context.
Would it be slow? Yes, just like "conscious" processing is for humans. Okay so today it would be much slower than humans process their sensory input, but in 10 years? 20?
As for how to represent an urge to reproduce within this paradigm - I'll leave that as exercise for the reader.
Two modalities down. Another couple hundred to go.
Unfortunately we're fast running out of the modalities that neural nets have shown capability in (image, text, sound... I think that's it).
In a discussion about whether LLMs could have agency and generalised reasoning ability, you suggested it was unfair because they hadn't received all the i/o a typical human did.
I pointed out that LLMs wouldn't be able to reason about that i/o (and if we made it fully fair and trained them on the comparatively small subset of text and discernible words humans learn from, they'd probably lose their facility with language too)
I don't disagree that bolting LLMs to other highly trained models like a program for controlling a robot arm and intensively training can yield useful results, arguably much more useful results than building a digital facsimile of a human toddler (toddlers produce pretty useless outputs but also have stuff going on internally we can barely begin to adequately replicate in silicon). But that isn't exposing an LLM to equivalents of human sensory input to get back an autonomous agent with generalised reasoning capacity, that's manually bolting together discrete specialised programs to an LLM as-message-passing-layer to have a machine which, given more training is capable of a slightly broader range of specialised tasks.
> What makes human intelligence different from today's AI is the ability to ask why, reason from first principles, and create experiments and models for testing hypotheses.
This is quite unfair. The AI doesn't have I/O other than what we force-feed it through an API. Who knows what will happen if we plug it into a body with senses, limbs, and reproductive capabilities? No doubt somebody is already building an MMORPG with human and AI characters to explore exactly this while we wait for cyborg part manufacturing to catch up.