This is not the revolution you think it is.
```python > 338383*887282 ... ANSWER ... ```
Yet skynet never came.
LLMs are not AGI. A human with below average intelligence is still a league above a chimpanzee. A chimpanzee will never be able to read, not because "it's too dumb", but because a chimp's brain lacks the actual hardware for reading. The LLM is the chimpanzee. The gap between an LLM and a "human with below average intelligence" is far more than 10x.
In which direction?
GPT-4 passes the bar exam with a top 10% score. How do you think a human with below average intelligence (or even with average intelligence) would fare?
Copilot generates programming code that solves problems, and in most cases the code is correct. It outperforms many junior professional developers. Do you think a human with below average intelligence could do that?
Absolutely, it's why Javascript is so popular.
Do you mean a programmer with below average intelligence (for a programmer)? Because I'm having a hard time believing that you actually believe what you wrote.
BTW, JavaScript is actually a fairly difficult language to learn and use. Python, BASIC, and even Fortran are much simpler conceptually and have far fewer pitfalls. JavaScript is popular because it's the only language that every modern computer has an interpreter for, not because it's so easy that idiots can use it.
I don't understand what rote memorization to pass a test has to do with intelligence. For the record Kim Kardashian passed the bar; I imagine anyone given the proper motivation and time to study could do it, it's not a hard test.
If AGI is a computer passing a test, then AGI was achieved a long time ago. I don't think a human with below average intelligence could multiple 2 very large primes but I don't mistake my calculator for intelligence. A below average intelligence human can drive a car with a couple hours of training, an LLM can't do that (with a far lower power budget as well).
I'm not saying AGI is impossible, but it's clear LLMs are not AGIs; it's not a matter of having a 100x more powerful LLM, just like making an Ape better at sign language won't make them better at abstract reasoning. An Ape's brain fundamentally lacks the mental machinery for higher level things that humans do. It's not a question of not being smart enough. LLMs are simply one component of the human mind.
AIs couldn't even pass a third-grade reading comprehension exam until about 5 years ago. Computers being able to pass tests designed for humans is a very new thing.
> it's clear LLMs are not AGIs
And the main argument for that is that "it's clear". They're beating lawyers, doctors, and software engineers, but obviously, that's not real intelligence...
Yes! Captchas so effective!
For instance, if it could make a peanut butter and jelly sandwich? Most challenging things that are easy for us are in the motor domain. While important, I think “intellectual AGI” is a meaningful milestone and closest to what most people think of when they think AGI.
The "G" in AGI is general. A computer program or system that could both, lets say write code and learn drive a car would be something closer to an AGI. Written tests are remarkably brittle in showing how intelligent someone is - like we already know the limitations of tests in the real world! Einstein famously flunked his entrance exam, but then invented general relativity at 26; but other posters would have you believe an LLM is more intelligent than Einstein because it could pass a test. When an LLM defends a dissertation in arguably any field that would be way more impressive than an LLM passing a test that humans already designed and know the answers to.
One problem I have with saying a 100x more powerful LLM could become AGI is that there is nothing that leads me to believe that LLMs, as they exist currently, are capable of synthesizing new knowledge and I'm not sure what breakthroughs you would need to get there. Once you start to think about that, you start to run up on the limitations of the LLM. If I were to invent a completely new programming language, I could probably teach a junior engineer how to use it in a week, but the jury is out if I would need to first generate 50,000 sample program and spend $1,000,000 in gpu compute to get an LLM to output the same thing. It's hard to consider such a system AGI. Further still, sure you have Google spending millions of FSD, but it's hard to consider the system they have as general. Could I take Waymo and have it pilot a forklift? Or a submarine? How much would that cost? A """below average intelligence""" human could learn to use a forklift in an afternoon.
All in all, there's more to intelligence than written tests.
This is a very problematic word. For example if you were a civil engineer and went "throw me any old design for a bridge, I have a river I need to cross", you'd have your license removed.
Intelligence is too massively loaded, and too much of a gradient even across humans to try to some up human or AI abilities. It is a multitude of different capabilities that don't necessarily have to be bundled together for something to be 'smart', 'useful', 'capable', and/or 'dangerous'.
> AIs couldn't even pass a third-grade reading comprehension exam until about 5 years ago. Computers being able to pass tests designed for humans is a very new thing.
You're narrowly defining what a "test" is. And therefore the "until 5 years ago" doesn't make any sense. A test is not just written exams like a "third-grade reading comprehension exam." Is playing chess against the world champion not a test? A trad AI computer program beat him 25 years ago, not 5 years ago. Is diagnosing patients with a particular disease better than the average untrained human not a test? An expert system did that 50 years ago.
Your point is not a refutation of the parent post.
I think we all can agree that even the best LLM currently is not AGI. That's not what being disputed here I think.
However a 100x more powerful LLM is not just 100x better at recall. A 100x more powerful LLM is not just 100x better at being stupid hallucinatory parrot. A model that is just 100x bigger is not necessarily 100x more powerful if you define power is the ability to achieve goals.
However pure language models will always lack something else: the ability to ground things in reality.
I recently had a dream where I solved some problems and when I woke up I realized that those solutions were bullshit, but I also realized the whole approach of my dreaming self was very similar to what a LLM would have done.
We only have our 5 senses to go off of. Meta has already put out one multimodal model incorporating multiple data types, openai is undoubtedly working on it too.
It's not a matter of which senses you have, but about being able to "continuously" use them.
The current LLMs are basically unfiltered raw thoughts that must be continuously refined. A similar thing happens in our brains and only a little bit of that is accessible to our consciousness
Exactly. But, AFAIK, it's also the part that does the bulk of actual thinking and decision-making for us. In that sense, LLMs may be closer to AGI than people expect, because they seem to be capturing the actual core of intelligence and reasoning - and the missing bits (like long-term memory and higher-level thought stream filter/censor) may be much easier to bolt on to them.
The output of LLMs is kind of like our stream of consciousness, there's a lot of things I think, then discount after internally reflecting on the thought which the often leads to a more correct solution. Having an LLM 'think' like this natively would massively increase the necessary the amount of compute needed, hence the expense, so at least in any public products it's not being done at this time.
I wish I had time to play with it some more right now. The pace of progress in the field is giving me a serious case of FOMO.
Balance, proprioception, hunger, …
Disagree, for the record. If I’d described the capabilities of contemporary AI to 100 AI scientists 5 years ago, I bet more than half would agree to call that AGI. Further, more than 90% would assume that these capabilities were decades and decades away.
This is hard to believe, the all you need is attention paper was 6 years ago, GPT1 is 5 years old and GPT3 is 3 years old. The current crop of LLMs wasn't something that happened overnight.
While I know the wider point you're trying to make, Kim Kardashian is a very smart business person. She may just not fit your narrow definition of "intelligent".
I believe you do have a good point overall, but this here is, IMHO, a rather bad example, because I'd argue GPT-4 is already capable of abstract reasoning, and this seems to be exactly the skill that improves with increasing dimensionality of the latent space.
I agree that LLMs are equivalent of a single component of a human mind. Specifically, I think they're closest equivalent to our inner voice / inner thoughts. But this part is arguably exactly the one that does most of the abstract reasoning (and most reasoning in general), so I think in fact LLMs do have the "hardware" for that specific aspect. What's lacking right now is the equivalent of the higher, "conscious" layer, that guides, filters and censors the stream of thought. That, and long-term recall. Short-term memory might atually correspond to what the context window is in LLMs.
That, and fusing in all the other senses (sight, sound, smell, taste, touch, time, etc.).
The reason scientific researchers in this area are using the term 'AGI' so much is that it does fit the definition of AGI ... *For some definition of AGI*. And there lies the problem - no one can really come to a good consensus on a good definition of AGI. This is why many scientists in this area are avoiding the question altogether - the question is loaded, and is misinterpreted by the public if statements are made.
So, for example, if I make the statement here that e.g. GPT-4 has intelligence which is general (AGI), it will likely be met with a rabid response from HN. However, the claim may be more dull than you're expecting. People often conflate AGI with things that are not required, such as agency, etc.
This definition from [journal Intelligence Vol 24, No1, 1997] can be that "some definition" of AGI: must be able to 1) think abstractly, 2) comprehend complex ideas, 3) reason, 4) plan, 5) solve problems, 6) learn quickly from experience.
Many of these GPT-4 can do, if some modifiers are allowed - for example, GPT-4 can learn quickly from experience so long as you aren't starting a 'new' GPT-4 system from scratch every time you want to interact with it. This is probably preferable, since much of the experiences it will have are personal to the individual working with it, and it would be highly undesirable to do the opposite here. Planning was difficult for the system early on, but it has appeared to learn that it is helpful to lay out a plan early on in a large task, so that appears to be a capability as well, at least on a basic level qualitatively. There is some good literature on ability to reason (and solve problems in abstract and complex ideas) from Microsoft's group on causal reasoning. In pretty much all tasks of causal reasoning the system can achieve near human performance, and LLMs as a category outperform previous SoTA from more targeted or specific systems made for causal reasoning.
Anyway, I would suggest to anyone that has strong reactions to claims about AGI to realize that they are likely building up the statement to be more than it is. Perhaps similar to 'machine learning' may have been misinterpreted years ago ("A machine can learn?! There is no tomorrow!!"), what is being stated here is often more narrow than you may believe.
Turns out that may be as much marketing as truth.
According to this paper[1]:
"although GPT-4's UBE score nears the 90th percentile when examining approximate conversions from February administrations of the Illinois Bar Exam, these estimates are heavily skewed towards repeat test-takers who failed the July administration and score significantly lower than the general test-taking population. Second, data from a recent July administration of the same exam suggests GPT-4's overall UBE percentile was ~68th percentile, and ~48th percentile on essays. Third, examining official NCBE data and using several conservative statistical assumptions, GPT-4's performance against first-time test takers is estimated to be ~63rd percentile, including ~41st percentile on essays. Fourth, when examining only those who passed the exam (i.e. licensed or license-pending attorneys), GPT-4's performance is estimated to drop to ~48th percentile overall, and ~15th percentile on essays."
[1] - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4441311
There was an article on HN yesterday debunking that. OpenAI is prone to exaggerating
But asking for a complete application nearly always fails, when an average human that knows programming should be able to make one that works.
"This talking dog is a dumbass. The risotto recipe he gave me sucked, and the C++ code he wrote is full of security holes. I don't see what all the hype is about."
Hint: ML will get better. Humans will not. ("Slow moving target," indeed.) What we're learning is just how many aspects of our vaunted 'intelligence' are really just features of our languages and the graphs they encode.
GPT-3 knows most programming languages better than I do, even though I literally learned to read with the Commodore 64 user manual back in the 80s and haven't stopped being a nerd since; and while GPT code isn't always correct or even compilable, it's not like I don't still make mistakes that cause compilation to fail a few times a day, and there's a reason we all insist on testing code rather than just assuming it will work when a dev stops typing.
Again, I acknowledge GPT-3.5 (and I assume 4 but have not used it) is not perfect: like others I'd call 3.5 a "junior developer" (and it isn't even that good in every subject!); yet, despite that, the only domains where I can regularly beat it at those where my perception of reality is fundamentally different from its perception (how words sound and how numbers are composed, so it's relatively bad at arithmetic and rhyme), or where it has been forced to be bad (ask it to write about conflict, my experience is everyone reconciles and lives happily ever after).
But in most cases it would take me years to get as good as it already is. And there are more of those subjects than I can remember the names of, too.
Very few people who "know programming" are actually capable of creating a complete application that serves a specific purpose. Many professional programmers struggle to implement basic algorithms like prime number testing. Coding AIs absolutely outperform the average programming professional, because the average programming professional can barely program. The software industry's demand for programmers is far too great for every coding position to be filled with top-class full stack engineers.
There's a lot of things you did during university classes. I bet you don't even remember half of them, and of the half you do, you couldn't actually do most of them from memory right now.
The advantage LLMs have over programmers is that they've seen much more code than any human ever would, remember pretty much all of it - not necessarily verbatim, but also as learned concepts, and that this knowledge does not decay. It's all equally accessible. Whether you ask LLM for basic or optimized primality testing algorithms, or how to use some obscure part of Microsoft's DCOM that approximately nobody on the planet wrote new code for in the last decade - they'll perform equally well. Not perfect, but not worse than an average software dev that does remember the things they're asked to work with.
Yes, and LLMs show all of the specific things you've listed.
https://github.com/drifting-in-space/botsh
Check out some agents that probably can already do more than you knew about.
Now, can and should all structure necessarily come from pure training? Is there a better "seed" structure that can make models more effective? Is there some other bits that are missing that are not in the microstructure but in the way the model is connected to the external world and itself (feedback loops etc)?
It seems to me that LLMs are perhaps capable of making up the inner thought loop, and the rest of the obvious systems seem doable. The context seems to be the hard problem; pulling in the proper things, and giving them the correct amount of attention.
This is likely one of the harder problems to solve. One of the strengths humans seem to have is the power of analogy. These analogies quite often can lead us into new paths of thought, or at least in the correct direction.
How do you do this in a machine system without wasting tons of power chasing dead ends, not sure?
Scaling can happen in two dimensions - model size and dataset size. What counts is the product of n_examples x n_parameters. That's why we have the super-Chinchilla laws, where n_examples >> 20*n_parameters. Scale the data, keep the model lean. Not to mention dataset quality - if you got clean diverse data you need less of it.
Another important trend is using LLMs to generate training sets. Example: ConstitutionalAI (pure RLAIF), TinyStories (scaling down LLMs), Alpaca (borrowed RLHF), AlpacaFarm - a recent paper promising fine-tuned models for $200 cost in 24h.
I don't think you are givin these "below average" intelligence individuals enough credit. What we consider a genius is the equivalent of a dog show obstacle course. We measure intelligence/genius as whatever is hard for humans and completely ignore what is easy because we fail to see the complexity behind the easy stuff.
Majority of human brain computation is spent on things that are simply not necessary for computer models (how to wiggle limbs, mouth, eyes etc).
Current LLM are impressive, but we know they can be much more efficient - we're using very low quality training data, we don't use any methods to question training input/evaluate it against current knowledge etc. Our current language models are based on reciting/memorization/force-feeding, not true learning.
Computer models have massive underlying advantage of working on CPU/GPUs where they can be modeled, cloned, retrained, have binary accuracy, can integrate with specialized code instantly, have access to massive memory storage, they are insanely fast and precise etc.
Looking at it from first principles, there is no reason to think that near optimal runtime should not be more efficient than human brain on currently available computers.
We don't need to simulate full brain just like we didn't have to create super fancy legs to go to the Moon.
Wheels are trivial but useless without bearings.
Bearings most certainly aren't trivial. All the things you listed further are dependent on bearings somewhere.
Evolution is a greedy, lazy optimizer, so it promotes things that work a-ok for a given environment.
It's also worth noting that wheels alone are not too useful for transportation, as they're only half of the picture. The other half is roads. That is, because we couldn't (and mostly still can't) figure out all-terrain mobility systems that could navigate diverse environments, we cheated and locally flattened the environment, to reduce the problem to that solvable by a humble wheel. Evolution can't cheat like this.
I can imagine evolution creating macro-scale wheels that can be healed, and/or able to survive long enough before failing to be advantageous - after all, bones and teeth can hardly be healed if badly damaged, and yet they last long enough to stick around as core design elements.
This is why I mentioned roads. Whether or not evolution could iterate its way to macroscale wheels, it wouldn't, because they'd be useless without roads. Legs may be more complex overall, but they're an all-terrain solution that can be incrementally improved, and every improvement step grants improved survivability.
In any case this discussion is going sideways, the point is that nature doesn't have monopoly on being optimal. This also applies to intelligence/learning/modeling something better than brain.
Because Mars is a simple and boring environment. Most of its surface, and especially the parts we target with rover missions, are effectively flat sheets peppered with rocks - a decent set of wheels and suspension is close to optimal for navigating such terrain.
Now, if we were to send missions to a planet that's mostly forests and rivers, like Earth used to be, then wheels wouldn't cut it - not before cutting down some of the forests first.
> the point is that nature doesn't have monopoly on being optimal. This also applies to intelligence/learning/modeling something better than brain.
Fair enough. Nature doesn't do globally optimal - but it makes things heavily optimized for their environment. That's why our planes are nowhere near as energy-efficient in flying as birds are, but birds cannot travel as far and as fast as our planes can.
Original argument was around optimising on intelligence and that biology doesn't hold best-possible trophy on it.
We don't need to match number of neural connections in human brain to exceed its intelligence.
That's also true because of wheels/road thing, in that we can "cheat" here too. More specifically, some of the neural connections in the human brain are dedicated to sensing, processing and controlling the dynamic state of human body. Purely-software AIs don't need those for intelligence.
Interestingly the latest Mars rover also includes a small helicopter, another technology which requires spinning something on a bearing and does not commonly exist in nature.
https://www.livescience.com/39577-insects-with-leg-gears-dis...
Carts, chariots, and wheelbarrows (to name but a few examples) have been useful for thousands of years without bearings.
Or more importantly - level ground.
Because roads don't exist in nature..... imagine trying to out run a predator if you just had wheels but no roads.
> Nature creates tentacles, not jet engines, nuclear energy etc
Under water I think you'll find there is jet propulsion.
Plants and animals are powered by nuclear energy - the remote fusion reaction is in the sky. Why have an internal nuclear reactor - fissile material is quite rare and fusion is quite hard to contain.
Human's are overly fond of their own children ( machines ) but fail to see the complexity of life. People are in awe of the latest Boston Dynamics robot - wow it can run - wow if can leap - and but don't post pictures of horses or gymnasts.
But that's just because the internet is already filled with cats
Technically, yes, but most people are going to be thinking of the spinning turbines of doom with the absurdly hot fire in the middle, and that isn't.
The reasons why biology didn't evolve what we discovered don't matter.
What matters is that it didn't, yet they can exist and outperform.
Similarly brain is not an upper ceiling for intelligence.
We can create more intelligent machines than us.
Things like energy efficiency don't matter as much - when we run large models, nobody cares that it may require more energy than two sandwiches and a beer per day. We do care about energy efficiency but not at scales of biology.
That's not to say we can't do this with computers and less computational power. However, it's really improbable that a couple layers of adaptability on an artificial neuron network will be anywhere near sufficient to simulate intelligence in even a rodent.
Wheels are not trivial in a biological sense. Topologically most life is either a tube or a cup depending on digestive systems. Wheels are separated from the body, which would be difficult for base cell division to produce.
Some animals like the pangolin are round shaped and do roll, but it just seems non optimal.
I’d say it’s curious nature hasn’t produced more creates that like making wheels like the dung beetle does, but nature made us and we do like making wheels.
A disk filled with random bits is "similar" to a disk that stores the whole Wikipedia's text content. So writing the whole Wikipedia is an effort of a hair's breadth...?
Sorry, can't help commenting the weirdness of high dimensional spaces. If you take an cube of a size of a hair breadth (100um) in a space of hindreds of billions of dimensions like gpt4, distances between random points in that cube are in the order of tens of meters...
Except that structurally the brain is clearly has vastly more capacity than the GPT-4 model.
So sure one brain doesn't look that much different to the other - and it's in the details of the learning, wiring.
But the brain, looks vastly different from a GPT-4 model in terms of capacity - with trillions of connections - with each connection and internal state being more subtle as well.
> vastly superhuman performance
In terms of specific tasks computers ( whether you write the program explicitly or it's learnt by tweaking params in a network ) have been there for decades.
So the question is really around which tasks can you apply computers successful to. Neural nets are allowing programs to be written that weren't possible be hand.
I find it amusing that people worry about ChatGPT etc al putting programmers out of a job, when it already has in the sense that CHapGPT is a program that was built by another program already.
I would not be so sure about "clearly" bit. Brain has 100B neurons x 1000 connections. GPT3 has 175B connections, but to implement operations used in those connections Nvidia H100 uses about 5M transistors, which in human brain would have to be copied to each connection because nature didn't invent software. (assumes one needs one full CUDA core to implement necessary ops, and that H100's 80B transistors are evenly divided between its 16k cores)
Also each neuron can hold much more state than a transistor.
For example they can respond to the timing of incoming events without having to build that capability with recurrent connections etc.
In addition the neural connections themselves have properties.
> because nature didn't invent software.
Eh? How do you explain the fact that brains can learn and aren't a fixed input, output engine?
Neural nets are software - just ones programmed by trial and error. Similarly for the brain.
What are you referring to?
> Also each neuron can hold much more state than a transistor.
But can it hold much more state than 8 transistors? 16 transistors? 32 transistors?
> For example they can respond to the timing of incoming events without having to build that capability with recurrent connections etc.
But they can't fire 3B times a second. Transistors though can accumulate a number and fire when it reaches a threshold much faster (although that is not really used in transformers). There are differences, but it is unclear how to quantify them. At least I don't immediately see how that feature is a definite advantage, and not simply an implementation detail.
> In addition the neural connections themselves have properties.
This is very technically correct, which is great because in this form it shows to be a non-argument. Individual transistors also have properties. Even molecules do. Which of those properties are essential for the learning process is the important bit, and it is quite possible that none are.
> Eh? How do you explain the fact that brains can learn and aren't a fixed input, output engine?
Eh? Brains learn by physically growing parts of neurons. That growing can not be moved around to another set of neurons. That property is essential to be deemed "software".
Hmm - that's quite a slow process - sure it happens - but I think you'll find that brains can learn on a faster timescale than the growth of connections would enable.
> That growing can not be moved around to another set of neurons, That property is essential to be deemed "software".
Not sure the ability to easily copy is the definition of software. Connections can be activated or attenuated without growth - that's the equivalent of learning your weights. Isn't that cumulative set of attentuations/activations software - ie you remap the input/output engine dynamically?
Fair point - but I'd argue the power comes for the combinatoric complexity, not the speed of operation.
So those timing effects can also combine with the simple connections - so if a neuron has 2 connections - the output can be controlled by not just the last signals for the 2 connections, but the relative timing of each of them.
Suddenly your input space you are operating over is much much bigger - you've got 2 inputs and hundreds of ways the timing could be adjusted. Massively bigger.
The fact that the whole thing doesn't operate in a big synchronous cycle is a huge jump in possible states.
LLMs can help humans develop new LLMs faster, but mostly in implementation (CoPilot and ChatGPT), and that's not really the important part. I have yet to see an LLM come up with original ideas.
Given that training data seems to be a big bottleneck, and LLMs are really good at generating text, I think that maybe we can start to talk about the possibility of "singularity" once LLMs are able to generate their own training data that increases their abilities. After all, humans are able to do this. That is the history of human knowledge.
Think of a human with below average intelligence. Then think of a human genius. Now consider how incredibly similar their brains are, despite the massive performance gap. It's not like one has 10x the number of neurons/synapses/connections etc. of the other. They're both healthy human brains, and you need powerful technology to even distinguish them structurally.
Considering this, it seems perfectly possible that a model like GPT-4 is just a hair's breadth away from vastly superhuman performance. It certainly beats the average human at many tasks already. The gap between a moron and a genius is a lot larger than that between GPT-4 and a superhuman.