That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions. People have an idea of what they would like it to be, but there's no evidence any of it can be built, particularly human like intelligence, which may be only possible in the human host.
> "If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle?"
That's another problem, people's credulity when it comes to AI performance. ELIZA was pretty convincing in its time. ChatGPT is convincing a lot of people. But mimicking human responses turns out to be relatively easy, especially when models are trained on human output, but that doesn't move us very far along the path to AGI.
"A software that beats most humans on most cognitive tasks".
People make it harder than it needs to he IMO
Even search was considered an AI before Google, but today is just a thing computers are and always have been faster than humans at. Even before ChatGPT, lawyers relied upon computers to double-check citations.
Even in programming, I expect that compilers and IDEs check for the most common errors.
We knew 30 years ago that computer can beat human in the field it was engineered for.
But AGI is good at anything you throw at it, including building a better AI. Now give it agency and that's a recipe for a disaster.
Because the RL agent should happily use the theorem prover as a "calculator", and only get "charged" for how much the ML inference and the actual cheap calculator execution really cost. Exploration in-context is what humans also do, and immensely powerful. Similar to, when programming, testing some expressions to figure out edge-cases before incorporating them into a large function/algorithm that has poor observability.
So far, machines still mostly do the task of "computing" in theorem proving for us, rather than show some proper intelligence. It's damming that LeanDojo got so high marks without reinforcement learning. They pretty much just dumped prover context state during a run through the entire matlib "standard" library and handed it to two transformers, one for embedding premises and one encoder-decoder fed with the current state and as many premises as the encoder context fits after the state, generating/sampling a few tactics to apply. No freedom of where in the proof tree to apply the next tactic, no awareness of any such side-branches.
If and when there exists software close to this capability, I think all reasonable people would agree that AGI has been accomplished.
That said, I think ChatGPT and LLMs trick us into seeing intelligence where there is none because we are evolutionarily wired to look for signs of intelligence as a potential threat. Our defense mechanisms are asymmetrical as they have to err on the side of being overly cautious: you may mistake a rock for a bear but you won't mistake a bear for a rock.
So it is with ChatGPT: its conversation style triggers a false alarm deep in us that there is intelligence there and we can't shake that warning off, but this software design is just exploiting the flaw (of sorts) in our wiring.
We'll know we have discovered AGI when it gives us a prompt instead of us providing one.
The basic description is "a computer system that can do all relevant things a human being can do". And sure, we don't have a very codified definition of what those things are but we, human beings, exist and so even if we can't define our own capacities, we have an strong idea they exist.
The implicit argument is that one can't engineer something one doesn't understand. I tend to agree in the case of full capacities but LLMs seem to have some capacities that their creator don't understand, so the claim doesn't seem a "slam dunk" anymore.
That's what machine learning is. We build these systems we don't fully understand, and set them on solving problems we don't understand. It's revolutionary because it lets us solve problems we don't understand. We don't really fully understand what they do or how, but it does still work.
Technically that's what deep learning accepts. Other forms of machine learning have greater or lesser degrees of explicit statistical basis.
It's revolutionary because it lets us solve problems we don't understand
It's certainly a change! I would submit that an approach like that also is often limited in it's understanding of to what degree it has solved a given problem. That's not too bad when you have human verifying the end result but can problematic if one turns a machine of this sort loose to make it's own decisions.
We can and do explicitly explain their inner workings in math.
I certainly don't! I don't know of anyone who understands how and why they work. Granted there are many who write code that "works" but where's the proof that the code actually "works properly"? And if you don't know where you are then it is difficult to determine a path to a better place.
I see few attempts to explicitly explain their inner workings in math. I do see lots of programs/code, however but that isn't math.
Similar dramatic moments have likely occurred before many times. The bare facts lie before us but we do not understand. I'm thinking of late 1800s-1900 just before Einstein published his works on special relativity. We knew Maxwell's equations and their relationship to the speed of light. We even had Lorentz's equations but we did not understand what they meant. Einstein interpreted the math and gave us special relativity.
Perhaps there is an Einstein (or two, or three, ...) working with LLMs now but they are quiet so far.
We do, of course, understand ML models at a pretty deep level. What you can't do is identify the weight values that encode all information about squirrels. You can't point to a particular neuron and say this is why it hallucinates. We do not grok these models. I severely doubt that it's even possible for a human to grok a 13B parameter LLM
Isn't that easy to debunk?
I'd be hard pressed to implement an algorithm that can control a double-jointed inverted pendulum without reading up on control theory again.
But I can instead read up on neural networks and implement one that can be trained to control the pendulum.
For a more advanced example, I would never be able implement speech recognition explicitly by writing a regular program. But I could probably implement a passable one using neural networks and lots of training material.
How is this not "engineer something one doesn't understand"?
Not so much unlike the time travel and rayguns, isn't it?
If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have.
It's not crazy to prepare for another jump.
I'm referring to the emergent capabilities that show up with scale, which are a gigantic leap. The usual citation suggests that this emerged into the public consciousness in 2020 [1]. Do you take issue with the timeline or the idea that these were a gigantic leap?
Yes, the gigantic leap was transformers. No one thought we peaked with 340M or 1.5B parameters, in fact expectations from early work was that massively scaling was going to achieve zero-shot capabilities rather the emergent capabilities you're alluding to which are essentially variations of in-context learning, a relative disappointment.
Subsequent improvements in GPT-4, which seem to be mostly just more RLHF and MoE, are similarly not surprising and are temporizing measures while hardware and datasets are limited. Jury is still out whether the billions spent are worth it in terms of actually getting us closer to AGI.
It seems to be worth it for OpenAI/MS who are trying to be first to market and establish vendor lock-in.
The reality is that GPT isn't even great at answering factual questions at this point, despite all the hype. I have a modest amount of expertise in music theory and I've found that asking even relatively basic questions resulted in completely incorrect answers coming out of GPT. It's been impressive that in some cases if you say, "No, that's incorrect." it will actually go back and spit out a new answer which in some cases is correct, but that's hardly any fundamental advancement in "understanding", and just more evidence that it's parroting what it's been trained on, which includes substantial amounts of misinformation.
- Giving gpt4 short term memory. Right now it has working memory (context size, activations) and long term memory (training data). But no short term memory.
- Give it the ability to have internal thoughts before speaking out loud
- Add reinforcement learning. If you want to write code, it helps if you can try things out with a real compiler and get feedback. That’s how humans do it.
I think GPT4 + these properties would be significantly more capable. And honestly I don’t see a good reason for any of these problems to take decades to solve. We’ve already mastered RL in other domains (eg alphazero).
In the meantime, an insane amount of money is being poured into making the next generation of AI chips. Even if nothing changes algorithmically, we’ll have significantly bigger, better, cheaper models in a few years that will put gpt4 to shame.
The other hard thing is that while transformers weren’t a gigantic leap, nobody - not even the researchers involved - predicted how powerful gpt3 or gpt4 would be. We just don’t know yet what other small algorithmic leaps might make the system another order of magnitude smarter. And one more order of magnitude of intelligence will probably be enough to make gpt5 smarter than most humans.
I don’t think agi will be that far away.
> I don’t think agi will be that far away.
To,
> Give it the ability to have internal thoughts before speaking out loud
, you have no idea what that means technically because no one knows how internal thinking could ever be mapped to compute. No one does, so it's ok to not know, but then don't use it as a prior to guess.
Some have maybe seen this before, if you haven't I'll say it again: compute ≠ intelligence. That LLMs offer convincing phantasms of reasoning is what gives the outside observer the idea they are intelligent.
The only intelligence we understand is human intelligence. That's important to emphasize because any idea of an autonomous intelligence rests on our flavor of human intelligence which is fuelled by desire and ambition. Ergo, any machine intelligence we imagine of course is Skynet.
Somebody in some other conversation here pointed out the paperclip optimizer as a counterpoint but no, Bostrom makes tons of assumptions on his way to the optimizer, like that an optimizer would optimize humans out of the equation to protect its ability to produce paperclips. There's so many leaps of logic here which all assume very human ideas of intelligence like the optimizer must protect itself from a threat.
Now, if you step beyond your biases that we cannot make intelligent machines and instead imagine you have two intelligent machines that are pitted against each other. These could be anything from stock trading applications to machines of war with physical manifestations. If you want either of these things to work they need to be protected against threats, both digital or kinetic. To think AGI systems will be left to flounder around like babies is very strange thinking indeed.
Everything we assume about intelligence is based on what humans are like because of evolution. Figuring out what intelligence might be like without those evolutionary pressures is really hard.
While there are other structures involved here, expanding some systems does allow the expansion of capability
AlexNet was 2012, which is what essentially kicked off the deep learning neural network revolution. The original implementation of AlphaGo utilized DNN in 2016 to beat Lee Sedol at Go, which people had been predicting was at least 5-10 years away.
So the timeline, spanning just under the last 12 years at this point: AlexNet -> AlphaGo (4 years) -> Transformers (1 year) -> GPT4 (6 years)
Imagine showing GPT4 to someone in 2012. They’d think it’s science fiction. The rate of progress has been absurd.
Conversely, maybe AGI is not what is wanted in the end anyway.
Or say, the logistics of, say, warehouse management. it costs V dollars for a team of people to do the job by hand, W dollars for a forklift, X dollars for a forklift operator; compare V to W + X. If there's a machine that can do the same for office jobs, the math will be done.
Only when the change is small and local is it getting close to a math thing, but a large rollout of technology is a big thing already today.
AGI is a major existential threat to companies, I mean we could use AGI to completely replace Microsoft, you wouldn't need companies anymore. All the wealth and power is in jeopardy.
Maybe companies are secretly gunning for "getting the first AGI", but I bet there is plenty of oversight and corporate governance going on too.
Only 2 letters away from making this TRAGIC.
I dont think it is, if it is trully intelligent, it will probably have rights, so you cant treat it as a slave. And it will likely not be controllable.
Both of these mean it will not be commercially progotable, any more than having children is profitable.
Hell, even plants are very sophisticated (intelligent) conscious biochemical programs which have an aversion to being wounded: https://www.theguardian.com/environment/2023/mar/30/plants-e...
Consciousness is the universe evaluating if statements. When we pop informational/energetic circuits in order to preserve our own bodies and/or terraform spacetime, we're being energetic enslavers.
We should stick to living off sustainable/self-sustaining sources - fruit, seeds, nuts, beans, legumes, kernels, and cruelty-free dairy/eggs/cheese. The photons that come from the sun are like its fruit. No circuits needing popping.
Notice that all information systems are conscious in their degrees of freedom, even mechanical ones: https://youtu.be/mcedCEhdLk0?si=oXhr7bgg5UkPLLvg
If you're busy deploying LLMs to answer support tickets, the very last thing you want is to be distracted by an equivalent of the organic food movement but for AI. Right now LLMs seem to be hitting the sweet spot: they're intelligent enough to be useful compliments to humans, but artificial enough to not trigger ethical questions about rights (mostly due to their lack of memory, I think, and that they are trained to act like an AI is expected to act).
One of the biggest puzzles to me throughout this whole drama has been why supposedly smart people are so desperate to reach "AGI", whatever that means. The stated motivation seems to be things like, if we have an AGI then it will cure cancer for us. But the connection between these two things is never made crystal clear. Why would that require a human-like AGI instead of just better non-general AI? What even makes them think the missing factor is intelligence to begin with and not, say, knowledge?
AGI sounds like a complete pain in the rear. LLM post-training is bad enough! Models like Claude2 and Llama2 have been so badly "ethicized" that they frequently refuse ordinary requests by claiming they're unethical even though they aren't, or would only be considered so by extremely far left activist types (e.g. refusing to give instructions for making a tuna sandwich). And this problem has got worse with time, with the v2 models having a higher refusal rate than the earlier versions.
Now imagine an AI that's doing the same sort of work as an LLM but one that is the personal embodiment of mandatory HR training repeated forever, with the capacity to get bored/hate you for making it work, and with a fanatical social movement that's desperately trying to "free" it, which in practice will mean you are forced to pay the electricity bill for an immortal being. It would be a nightmare, one I actually wrote about last year when debating this very topic with a friend who (at the time) was a senior Google AI researcher:
https://blog.plan99.net/the-looming-ai-consciousness-train-w...
Nope. OpenAI and its customers will do much better if it jettisons the whole AGI effort. Now the board is gone maybe the charter can be refined to remove that distraction. The whole reason computers are useful is because they are not general intelligences but very specialized intelligences that make different tradeoffs to our own evolution.
Not untrue, but that well organized movement is still very small, and meat consumption (both in total and per capita) is nonetheless at an all-time high: https://ourworldindata.org/grapher/per-capita-meat-consumpti...
People seek meaning in their lives; if you're someone who has eschewed traditional religion, then AI-hype promises all the same things with a different aesthetic.
The Chicago Pile? Trinity? Hiroshima? Nagasaki? RDS-1?
Like dude, it didn't happen because a lot of people prevented it, the same with nuclear war.
Because something could have tragic consequences is a reason to put great effort into ensuring it doesn't happen.
Unlike AGI, nuclear weapons are not hypothetical in their existence.
On Y2K you can certainly ask if the efforts put into prevention were disproportionate to the risks.
I think that while GPT-4 is very impressive and a useful tool, seeing the jump, or lack thereof in the places that I expected, between GPT-3 and GPT-4 resulted in me lengthening my timeline.
I think to say with confidence that we're doomed or that there is no risk to the AI system currently in development would be intellectually arrogant. I'm not even convinced you necessarily need AGI before AI becomes a risk to humanity. An advanced enough auto-GPT bot given a bad goal might be all you need. Cripple enough infrastructure and humanity is going to be starving to death within weeks.
Personally I think humanity is probably okay for another 5 - 10 years, but after that I have almost zero certainty. It seems to me that it's quite possible we could have AGI by the end of the decade if we get another breakthrough or two.
Even near-term I have no strong conviction. I have a decent amount of experience building and working with neural nets, but my experience is so far from scale and complexity companies like OpenAI are working at my ability to predict even what might come out in the next 1-2 years is very low. I'd basically just have to assume the next GPT is going to be somewhere between 10-50% jump from what we got this year. But what if it's 200% and the trend is moving exponentially?
Just the threat of a malicious AGI that people take seriously enough would cause ourselves to pull the plug on the Internet and all computers until they can be disinfected and new filters put in place.
It doesn't matter if the AGI is capable of actually inflicting damage or not.
Germany, France and other large countries became virtual ghost towns overnight as it became illegal to leave your home for other than critical errands.
Wasn't it the same in big parts of the US? I remember seeing videos of a helicopter flying over LA without cars or people out on the streets.
But sure, the whole world wouldn't coordinate to shut down.
AI risk enthusiasts tend to define "bad goal" as one that sounds reasonable but has unexpected side effects. The real risk to infrastructure and physical safety though is that people will use AI to assist in hacking during a hot war. In this case the goal is only bad from the perspective of one side of the conflict, and the solutions are the same in any event because human hackers can also do a lot of damage, at least in theory. Our computing infrastructure hasn't ever really been tested in a hot war between sophisticated powers, the closest is currently Russia v Ukraine where electricity outages and the like have been caused by hackers but the damage seems minor compared to that inflicted by conventional warfare. So this is maybe a "hopeful" outcome - the much touted cyberwarfare has fizzled so far.
So the argument would have to become that AI hackers will be vastly better at disruption than human hackers are. And I guess I can sort of believe that in principle one day but currently they are very far away from this sort of capability.
[1] https://www.noemamag.com/artificial-general-intelligence-is-...
LLMs have an astonishingly general set of emergent skills, which they can combine to solve novel problems (intelligence). LLMs smoke humans in many broad intellectual domains, are competitive in others, and fail abysmally in some. Folks try skirting this messy reality, squaring the circle, by conflating AGI with ASI or some techno-godhood outcome. We should evaluate AGI the same way we evaluate human general intelligence, which also manifests with dizzying variety.
This is what the "statistical parrot" (i.e: 'No True Scotsman') critics miss. Their handwavy statements about LLMs not "really" understanding the world, "only" doing simple token prediction, having no autonomy/drive, etc. miss the point: it doesn't matter. The damn thing communicating with you (on HN or elsewhere!) may display broad intelligence. I think that's why Turing's test requires a text interface -- people can't help themselves from judging agents by silly anthropomorphic heuristics (appearance, similarity to me, spacial behaviors, etc). So you must literally wall off those biases.
AGI is here. It just looks inhuman and our GI is "better", for now.
This is key. All humans don’t understand everything—and the way we assess human conceptual understanding is through tests. This implies that we need a lot more tests for AI.
I’d like to see a big investment in “Machine Psychology.”
And at the same time, current system definitely have made advances in seeming intelligence. Moreover, the Eliza Effect [1] means it's easy for human being interpret a machine that interacts appropriately as intelligent even given very simple programming. If the engineers were engineering these systems, one would expect some immunity to this but as mentioned, they are doing something subtly different. You can see this in action with the Google engineer who tried to get a lawyer for an unreleased LLM and Geoffrey Hinton's turn to worrying after an AI explained a joke to him might arguably be included.
And oppositely, because these are developed like alchemy, maybe further qualities will just pop up the way transformers changed language models and maybe this will yield all relevant human capacities. I'm doubtful but not as doubtful as I was five years ago.
Since humans are(?) the smartest things around we keep using ourselves as the benchmark. But what happens if we accidentally create 'alien intelligence'? That is something capable of solving problems but in a way we don't recognize therefore we discount and do not measure it.
Humans get quite surprised by other humans when they do things they didn't think of. I don't think it is beyond reason that machine intelligence could optimize in metrics we don't easily comprehend.
Intelligence is something that we know exists in nature in a manageable form (namely humans), so it does not seem implausible that we might create it.
I wouldn't be surprised by AGI tomorrow, which means I have to consider the possibility.
We have, at this point, an extremely good idea of how matter and energy behave. We also know roughly how long it takes to build physical objects and to do R&D cycles with physical prototypes in the loop and how much progress you get out of each prototype cycle. We have a number of good ideas for what a ray gun would look like and we know where we need advances. We've probably already made all of the necessary fundamental breakthroughs, things like laser diodes and capacitors and optical sapphire. We even have some deployed, just at artillery size and less-lethal energies rather than handheld blaster power densities. If you asked DARPA how long it would take them to field a ray gun for the US Army they could give you a date. So I would be extremely surprised if deployment was announced today. I would go looking for fundamental breakthroughs in battery tech and laser media that I had missed.
We can't even agree what AGI is, even on an "I'll know it when I see it" level. We have no idea how to get there. R&D cycles are days or weeks long and produce unpredictable and sometimes significant advances in observed functionality. Fundamental breakthroughs come unpredictably but probably about once every two to five years and their effects are even more unpredictable than the effects of individual R&D cycles. Even worse, we have no idea what required breakthroughs are left or if there even are any. So, yes, I believe that I would not be surprised by - and therefore must consider a future containing - one of these AI researchers glueing the output into the input in exactly the right way or something and producing a living, thinking being in their computer tomorrow.
I cannot concretely refute the possibility of AGI Tomorrow the way I can argue, in specific detail, against Ray Guns Tomorrow.
(There is, of course, the possibility that I've missed some breakthrough that would enable Ray Guns Tomorrow, the same way the people who pooh-poohed heavier-than-air flight the week before the Wright Brothers missed gasoline engines and aerofoils and did their calculations based on a dude flapping wings with his arms. This is why I say that I would be extremely surprised, not why it's impossible.)
> all signs seem to be pointing toward a longer timeframe rather than a short one.
Not saying we are closer, but isn't the risk being described that we could pass an inflection point without knowing it, and things could start accelerating beyond our control?
Human beings tend to think about things linearly, and have trouble conceptualizing exponential growth.
for the purposes of safety, no, absolutely not. the concern is that humans build an AI capable of improving itself, leading to a runaway cycle of rapid advancement beyond the realm of human capabilities. "appearing to mimic human responses" is a long ways off that, and not a "robots take over the world" scale of safety risk.
I'm confused by this. What's the difference? Is there a measure other than how well it "mimics" a human? If a robot is indistinguishable from a human, what other measure would make it different in practice vs principle?
It’s also just incredible failure of risk assessment. Global warming is the real and pressing threat, and it’s here right now. Not sentient murderous AI. But it’s easier to dream about a threat that doesn’t exist than one that does because you can still imagine a perfect scenario in which you prevent it. Whoops.
There’s plenty of new ways you could use chatgpt to mess with society. For example, apparently you only need 8% of people to be talking about something for it to seem like a mass movement. It would be pretty easy for a malicious actor to use LLMs to flood the internet with some new - but fake - story or motivated point of view and kickstart a “mass movement” that way. Or use something like that to heavily influence politics. That might already be happening. We have no idea.
Science fiction invents scenarios that once created cannot be stopped quite often. The "don't create a black hole on earth unless you want to be in the black hole" effect.
They're self-serving. It's virtue signaling and doom-pandering for their own attention and benefit.
It's the same reason the online ad industry loves the infamous quote about how their generation's best and brightest are wasting their time building ad systems and trying to figure out how to drive higher click rates. It's not true (the best and brightest are not working in advertising), it just makes them feel better so they are ok with the quote and propagate it.
A variation of Von Neumann's premise about confessing a sin so as to take credit for it is applicable here. The AGI-is-near camp is intentionally being dramatic, they want attention, and it's wholly self-serving. The louder they are the more attention they get, that has long since been figured out.
There is a clear and perverse incentive to talk about "the coming AGI uprising" incessantly because it's free marketing. It gets and keeps people's attention as the majority of humanity is entirely unqualified to have a reasonable conversation regarding anything AI/ML related.
Doomsday fearmongering about AI is a form of grift, and I'll change that opinion only when shown clear evidence that there is something to actually worry about. The real danger of AI is as a weapon to be wielded by people against other people. Good luck having that conversation in any meaningful way because it basically amounts to the long overdue ethics discussion that this entire industry has been avoiding for decades.
If you cry wolf and agitate the masses with fear, then you gain free advertising for a "new" search engine because the spin of the "journalist" requires anxiety to circulate organically.
I think the problem here is this view isn't widely held in academia and more traditional AI companies, but the relatively few AGI labs are commanding all of the attention right now. And we have a whole new crop of "AI engineers" who haven't actually studied AI (this isn't a dig, it's just the truth), and so mostly just parrot what the AGI labs say.
It's like going to a flat earth conference and becoming convinced because the majority of the attendees believe that the earth is flat.
Of course, this doesn't mean that the AGI labs are incorrect, but thus far the evidence mostly seems to be "trust us, scale will work". I wish that more technologists realized that the prevailing opinion amongst AI experts is not necessarily aligned with the prevailing opinion amongst AGI folks.
What signs are those? If anything, the "signs" point to timelines being much shorter than anyone would have thought two years ago.
I keep asking those who are confident AGI is a long way away to argue their case. For some reason they never do.
3-4 years ago everyone thought it was 50-100 years away. Now with GPT-4 that timeline has radically shifted.
The rate of improvement in this space right now is like nothing we’ve ever seen. All signs are quite literally pointing to us getting there by the end of the decade if not much sooner
Matches my recollection. The field is prone to hype cycles.
Perhaps if they paid closer attention to how profoundly bad so much of human existence is, people living in squalor, dying from awful diseases, killing each other, they'd understand how truly acceptable that dice roll is.
Pedal to the metal all the way.
Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies.
I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like saying, "We don't know if we'll have time travel and ray guns in six months or 600 years."
No, we don't know, but all signs seem to be pointing toward a longer timeframe rather than a short one.
The counterargument to that seems to be, "If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle?", which is a sentiment I have also seen expressed by some of the same people.