I think the regulation of the feedback loops required for sustained, deliberate thought, and interaction with the world, will be the most difficult piece of the AGI puzzle, and might not exist today.
New ideas are "hallucinations" of our "existing data", that we eventually get around to proving. An AGI will require these "hallucinations", inhibition and excitation of them. I think it's going to be a tricky balance [1], for sane output.
1. "Creative minds 'mimic schizophrenia", https://www.bbc.com/news/10154775
There's a sibling talking about internal memory. That's how our brains solve the issue. AFAIK, nobody knows how to train something like it.
GPT-4 can divide large tasks up into logical steps and instructions that can be worked on independently by other instances, and it can create "compressed" condensed explanations of it's own state that can be stored externally and repeated back to it, or passed back and forth between instances. With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on.
Also that means you can just iterate with different contexts until you can deal with the problem. How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?
Perhaps the unknown unkowns?
I envision a scenario where a super intelligent AI seemingly runs off the rails and becomes obsessed with overtly complex problems that are totally intractable to humans. Where we'd be like ants trying to make sense of cell phones.
I think something like this could be hacked together, with summarization and parallel "focus threads", containing prompts that focus on details. These could be pruned/merged back together, to add to the "summary" higher level abstractions.
I use this approach already, to some extent, when a conversation gets too long and I need specific details. I'll start a new prompt, include a high level summary, and then "focus" on specific ideas to get more specific answers about a particular detail.
Seems that for any task where you can generate an error signal with some information ("ok, what you just tried to do didn't succeed, here's some information about why it didn't succeed"), GPT-4 can generally handle this information to fix the error, and or seek out more information via tools to move forward on a task.
Only thing that's really missing is somebody to crack the back of a memory module (perhaps some trick with embeddings and a vector db) is all it takes for this to be AGI.
Here to say I agree with embeddings and vector db for memory systems. However also to disagree this can lead to AGI.
I need to be able to save something to a variable and call back that exact variable. Right now, because it’s just pure text input of the whole conversation, it can forget or corrupt it’s “memory”.
Even something like writing a movie script doesn’t work if it constantly forgets characters names or settings or plot points.
To add to your comments, I suggest that local vector store of embedding vectors of local content is how I’m going with memory issue. Langchain like. That way all previous progress is tracked and retrievable. That way the system can retrieve all it’s “memories” itself. The recursive multi agent pattern is big deal in my opinion.
Please define this. ;)
[1] - https://www.pinecone.io/learn/langchain-conversational-memor...
I'd like the "stochastic parrot" crew to explain how interactions like this somehow don't involve the extraction of and manipulation of abstract meaning.
H: translate the following sentence to instead use olympic curling related metaphors ( do not explicitly mention curling, and do not use similes in forming the translation ): man:"I've been spinning my tires all day without getting anywhere" woman:"running out of gas?" man:"nah, I'm gassed up, geared up, oiled and revving. I just can't seem to get in the right lane, you know?" woman:"anything knocking?" man:"nah, I'm tuned just fine. I can't seem to get my GPS to connect, you know?" woman:"take a few laps without a destination, see if you can figure out the lay of the land" man:"you know, that might help. thanks barbara"
R: man: "I've been sweeping the ice all day without hitting the target."
. woman: "Losing momentum?"
. man: "Nah, I've got the energy, the technique, and the focus. I just can't seem to find the right path, you know?"
. woman: "Anything coming in your way?"
. man: "Nah, I'm on point with my aim. I just can't seem to get my strategy to align, you know?"
. woman: "Maybe glide a few times without aiming specifically, see if you can figure out the ice conditions."
. man: "You know, that might help. Thanks, Barbara."
It seems obvious to me that it is understanding the metaphors into underlying meanings to be able to translate them like this. I think half the pushback is just ego driven nonsense, similar to the sorts of people that make similar claims on the capabilities of animals.It's all just an argument that "boats can't swim".
Why are you so confident in this? We don't really know how human "intelligence" works. Whose to say it's not doing something similar?
It's a bit like saying an audio file doesn't contain "music" because it's just a binary representation of the original work.
Or cepholapod intelligence. Or magpie/raven/crow intelligence.
If I ask a chat LLM what its favorite flavor of ice cream is, it will select a statistically likely answer to the question.
That's the difference.
For reference: my favorite ice cream flavor is cake batter, but only if it's the one at Cold Stone. If it's from somewhere else, I go with cookie dough.
I would even argue that humans are more similar to that than we think. The main difference is that one of the characters is locked to the surface most of the time (not even very perfectly).
Why do you prefer cake batter ice cream? Because of some sequence of events leading up to present point in time (not unlike training), and/or random genetic variations (not unlike setting “temperature” parameter in GPT to >0)
They don’t claim that it doesn’t. What they point out is that humans have a tendency to ascribe intent and agency to the text it outputs. But the LLM is optimized for prediction, not survival, unlike humans[0]:
> Text generated by an LM is not grounded in communicative intent. […] Our perception of natural language [is mediated by] our predisposition to interpret communicative acts as conveying coherent meaning and intent
Some of the dangers they raise associated with this is that it will not realize that words it chooses are PII, dangerous to give to who they are talking to, or biased in a way that can cause societal harm:
> If the LM or word embeddings derived from it are used as components in a text classification system, these biases can lead to allocational and/or reputational harms. […] A Palestinian man [was] arrested by Israeli police, after MT translated his Facebook post which said “good morning” (in Arabic) to “hurt them” (in English).
What they encourage is to view it as a tool and assess how it may fail. For instance, you might object that the MT was simply incorrect; but realistically, sentences can be translated in many ways and with many intents (eg. Allah Akbar has a lot of contexts!) and the LLM may not be given the full picture of the situation.
The stochastic parrots paper is heavily misrepresented or used by people that don’t seem like they read it. For instance, it was cited in a recent petition asking to stop work on powerful AI, prompting a response from the authors[1] pointing out that it misunderstands the paper and emphasizing that they disagree with the petition.
[0]: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922
[1]: https://www.dair-institute.org/blog/letter-statement-March20...
I listened to a lengthy video on the "stochastic parrot" from the Alan Turing Institute yesterday after having made that comment, and it was mostly on the topics you mentioned, subtle and overt bias, etc. I kept waiting on them to explain the "it's not real" bit that I've seen implied by the phrase where I've seen it used ( or perhaps, simply read into it myself ), but it never really came in. At the very end, one commentator made a comment implying it's not really all there and the next responding didn't disagree outright, but you could see a kind of "well yes, but actually no" kind of expression go across their face as they talked around it.
I think the "stochastic parrot" idea is too harsh, but it does reflect that GPT is fundamentally incapable of looking at a small block of text, turning it into a set of logical facts, and recognizing if those facts are consistent or not. I think it is something that is possible to build with a refinement of GPT, but I'm not sure it can be done simply with glue code.
An AGI, by definition, can start with zero information, and learn on its own. Where are the models for that? The closest thing we have is Mu Zero, but we have to give it an objective function, which destroys the whole idea of AGI in the first place (as it should be able to generate its own objective functions) and it takes a shitload of resources to run for a very limited rule set. Until you have something like this in place, no version of Chat GPT will even come close to AGI. The problems that chat GPTs are solving are turning out to be not as complex as we thought, thats all that the "sparks of AGI" are.
What will happen assuming the current course of iteration on the models is that humanity will get is essentially better and better software that can solve a large range of complex problems. You will have essentially a large body of human knowledge that is compressed in some form, with the ability to interpolate between branches very efficiently. This will create issues because you will now be able to ask multi modal questions like "what do I need to do economically to ensure ultimate compliance of the population?", and the solution will involve things like which government offices to target with what information, generated videos of propaganda, special products that promote addition or some other health changes, e.t.c.
To prepare against this, the solution isn't to lock down the models and hide the key while telling everyone how AGI is going to kill us all, the solution is to allow everyone access to this and let the AIs fight against each other and reach an equilibrium.
I am afraid to explain this, because I have tried a preliminary version of it that I supervised step by step, and it seems to work. I think it is obvious enough that I won't have been the only one to think of it, so it would be safer to put the information out there so people can prepare.
I see a big disconnect on here between people saying GPT-4 can't do things like this and is just a "stochastic parrot" or "glorified autocomplete," and people posting logs and summaries of it solving unexpectedly hard problems outside of any conceivable training set. My theory is that this disconnect is due to three major factors: * People confusing GPT-4 and GPT-3, as both are called "chatGTP" and most people haven't actually used GPT-4 because it requires a paid subscription, and don't realize how much better it is * Most popular conceptual explanations about how these models work imply that these actually observed capabilities should be fundamentally impossible * Expectations from movies, etc. about what AGI will be like, e.g. that it will never get confused or make mistakes, or that it won't have major shortcomings in specific areas. In practice this doesn't seem to limit it because it recognizes and fixes its mistakes automatically when it sees feedback (e.g. in programming)