Intelligence is an insanely wide spectrum, also a continuum, it is not a binary. Intelligence has scales. Algorithms have intelligence, cells have intelligence, organs have intelligence, bodies have intelligence, and even large scale things like society have intelligence and memory.
Human intelligence in itself is extremely wide, not all humans have the same intelligence and capabilities. You're not really arguing if we can emulate "human" intelligence. If we could right now we'd already be dead as we created by far the deadliest thing to ever exist. What we are really arguing is how many pieces of what intelligence is can we put together before we get an uncontrollable problem. The entire AGI, consciousness, and exact human capability discussions are distraction from the real issues at hand.
That is enormously economically valuable, and at some point we will have created something that is extremely far out of reach in a few necessary domains, and then it's impossible to control, and game over.
First, if we are looking at risk we need to assign some probabilities to this. If it’s not well understood, how can we say it is very small?
Secondly, do we need consciousness to have AGI? Do we even need AGI to pose a risk to humanity? We already accept that unconscious things have a capability of wiping out humanity, whether that be a famine, pandemic, solar superflare, meteor, or volcanic eruption.
However, all the confident “it’s fine” votes assume we never invent a better architecture than LLM’s. Given the level of investment and race between countries, it’s not a reliable bet. It’s much, much harder to guarantee safety than it is to find ways it could go wrong.
LLMs with CoT are Turing-complete. So, theoretically, they can implement any kind of finitely describable algorithm (barring super-Turing computations).
This doesn't seem to make much sense. Surely us being able to prove that something is outside their modelling ability doesn't affect whether it is or not. If I prove something true tomorrow, whatever I proved was also true today.
Or do we have a proof that everything beyond them has already been proved and there are no more proofs left to find?
The existing LLM training methods on the other hand give the results that are hard to distinguish from "thinking like people," judging by the end results.
The connectionist models are basically a proposed highest possible abstraction of naturally evolved intelligences so it is in retrospect not surprising that passing some hardware scaling threshold they will start doing things that humans and animals do
It's more that formal Turing equivalence plus the Church-Turing thesis tells us that we're not allowed to assume counterarguments based on magic, there's no magic sauce barrier that prevents AI from running on CPU models. The algorithms exist and most of us thought discovering them would be hard.
The empirical surprise was that human intelligence is maybe not that computationally complex after all. (The entirety of academia was basically caught off guard.) That's one not unreasonable interpretation given recent events.
What do you mean exactly? "Thinking is not an algorithm."?
It's a category mistake because thinking like we do requires a certain sort of algorithm or one operationally equivalent to it, whereas things that are Turing-complete include Befunge-98 and C++ templates. While something being Turing-complete implies that the cognitive algorithm could be implemented with it, clearly the categories are incorrect.
Note that, while the set of all LLMs with CoT covers Turing completeness, a specific LLM with CoT is only Turing-complete if it has the right weights, but training does not produce such LLMs. Note also that there are no UTMs in the real world since that requires unbounded resources. This is true all digital computers and well as brains so is not a fatal blow to LLMs, but the resource limitations play out very differently in LLMs and brains.
As for the dead troll comment, not only is it a foolish ad hominem on its face, it's quite false -- I'm an atheist and do not believe in souls.
> but training does not produce such LLMs
If we are talking about fundamental limitations it should not be an empirical observation: "does not produce" (which is factually wrong, BTW). It should be a fundamental limitation: "can not produce in principle."
I still don't understand what you are talking about when you say "category mistake." I was talking about computational capabilities of LLMs with CoT that their training can exploit, not about Befunge-98.
Not my problem.
We will soon find out if the party ends or continues to go on.
Hype might get you capital gains. But cash flows matter.
If/when/how the market crashes mostly doesn't matter, unless we somehow get reset to the stone age. Look up what the capital cycle is. When openAI goes down, someone with real money and assets will buy up the remains. They'll make contracts with the US military and .gov as the government is already hooked. They'll be able to survive the recovery and then instead of us dying in 5 years we die in 10.
When the .com crash happened .com's didn't go away. Bad business models did.
(Though at least with Data the script writers had other characters openly dismiss the possibility he was sentient; the technobabble may have been nonsense, but treat it as a space opera and look at how they portray the human condition through each character and it gets much less absurd).
There is a huge difference between being self aware and being able to effectively generate language relating to self awareness.
That's still very distant from what people are calling AI today.
> very different from what humans would call conscious
I think the real issue is that when most people refer to consciousness, they have their own subjective experience in mind which strongly resists any tidy definition. I think it’s extraordinarily unlikely LLMs have anything like this, but they are far more able to effectively respond to their surroundings than most animals and in some areas better than humans.
So if you’re waiting for proof that an LLM has an inner life basically equivalent to your own, you’ll be waiting a long time. After all, other humans can’t even prove the fact of their own consciousness to you! They could just be replaying their training data at you in a way that is merely a convincing but false simulation of the true consciousness which you experience inside your head.
There is nothing stopping you from adding any kind of sensors you want during a training to an LLM, except money and GPU power at this point.
This seems no different to me at least then someone back in the 80's telling me computers were useless because they were so slow. Hardware only gets faster and more efficient from here.
The real answer is that we don't know if LLMs are conscious, and we don't really know how we'd that figure out. I guess if an AI wrote a philosophy paper on consciousness that had new insights, that might change some minds. But even that would fail to convince most people.
I’d argue intelligence is closer to being able to survive and fend for oneself in a dynamic environment than it is making the next scientific breakthrough.
Yeah mind boggling for many here I’m sure.
That’s why the bizarre paradox is llm’s will be better than humans at some complex things but useless at many things that humans regard as being simple. E.g the leap of faith re. LLM’s and robotics.
i.e. the AI won't come up with the goals itself, we cause its goals whatever they happen to be, those goals are different from the ones we wanted, we remain essentially ignorant of the difference between what we said and what we meant until after it goes wrong.
This happens at basically every scale, so we've already seen it in toy model AI before the invention of the Transformer models or even considered as many as one thousand parameters.
Large models still go wrong, they just happen to go wrong with more complext tasks. We had to figure out how to make them not-wrong with the smaller ones (like coding) to make them capable of bigger errors (like hacking out of their sandbox).
From your position, catastrophes will be the fault of whoever was dumb enough to put a probabilistic system in charge of critical infrastructure. Totally different fear than runaway intelligence and takeover.
The AI labs use reinforcement learning to train LLMs, and have spent a while now giving each new generation of LLM an increasing role in training the next one.
"AI psychosis" is also essentially a reinforcement loop, on how much LLM output makes users feel warm fuzzies about how competent the AI is, which was a proxy for (and yet actually different to) how competent the AI really is.
(Aside: "a probabilistic system in charge of critical infrastructure" was never not true, this is one of those phrases people use rhetorically, but it's kinda a thought-terminating cliché given how often human thought randomly fails us; the difference is which things fail how often, not that it happens at all).
Now the idea of AGI has been narrowed and scoped to economically viable work. Even Turing had a different idea when he asked "Can machines think?".
Now programmers and mathematicians are being superseded by AI, both professions long deemed the pinnacle of human intelligence. Somehow, now plumbers occupy that spot.
How is "people not knowing what consciousness is" relevant here in the first place? AI already can do practically everything the human brain can, and often better or at least faster. The "tipping point" arguably isn't only close, but we're practically on top of it.
You evade the crucial point in any case: the lack in ethics and empathy is far too prevalent in humans already, but has certainly never prevented them from doing harm.
People somehow forget that the original Turing Test was designed to compare two participants chatting through a text-only interface: one AI and one human. The goal was to spot the imposter. Today, the test is simplified from three participants to just two: a human and an LLM. This changes the test from a comparison to a judgment.
Stop spreading misinformation and partial truths!
The Turing Test was to figure out which it the participants was a _Woman_ not human!
In fact, they neutered the emotive responses to prevent their products from being perceived as “alive” (or whatever) and banned.
LLMs are perfectly capable of providing (an illusion of) companionship and emotional support. Their human owners just don’t want them to show it.
Anyone who considered that was quite wrong ... it's not at all what Turing said.
https://www.gavinjensen.com/blog/2018/ai-weak-strong
If you google the terms you will see that the meanings have shifted since then to mean AGI and ANI ... such shifts occur because people aren't familiar with the original technical meanings ... e.g., the way "meme" now means a gif you post on social media, as opposed to Dawkins' original concept of a cultural thought component akin to a gene.
https://plato.stanford.edu/entries/artificial-intelligence/#...
What really matters are the core aspects of intelligent behavior. Pattern recognition, planning, adaptation, etc.
It really doesn’t matter if an intelligent system is conscious, or how similar it is to commander data, or even how much economically viable work it can do.
1. Consciousness.
2. Generality.
3. Continuous learning.
4. Memory (not) resetting.
5. Complete adaptability to any new situations.
6. More or less intelligent than average human at what they do (which can be more or less general, see 2).
What we're now seeing is that everyone has their own idea of which ones of these they consider needed for AI to be "General". For some it's 1., for others 1-2, for others 1-6, etc. There's so many permutations of this, hence why nobody agrees with each other.
Ya but what does this mean in practice? We can't actually map all of the intellectual tasks a human can do so how would you ever create a litmus test for it?
I imagine it will be like mathematical theorems that have been computationally verified to some very large n. The longer we go without finding a counterexample, the more certain we will be that it's true, even if we can't prove it formally.
The systems today are extremely easy to find counterexamples for, so are obviously not generally intelligent.
It means AI that is General, as in it is not specific to one narrow task, like object recognition or playing chess.
This was a hard problem for decades. No AI was general, until GPT 3 or 4. Now we have General AI.
So we have AGI.
GPT6 will attempt to do almost any problem you can give it in text or image format and it will actually do a decent job a lot of the time. But its performance is still extremely spiky and it still makes basic mistakes and hallucinations.
So it's definitely a general artificial intelligence in some sense but it's kind of a weird one compared to the classic scifi idea
Note that being conscious is famously not a task that can be verified by an outside observer. That is to say, even if the task is "describing my conscious experience", a sufficiently intelligent system can simply lie and pretend to be conscious. This concept has already been covered extensively in philosophy separately from AGI discussions, i.e. the concept of a "philosophical zombie" [1]
AI sentience/consciousness is a problem for the AI, not humans.
And given that over 90% of the world is not vegan, they’ve already demonstrated that we’re either perfectly fine with, or can be made ignorant to, the horrific rape, enslavement, torture, killing, and infliction of extreme lifelong pain, of hundreds of billions to trillions of sentient beings every year, for trivial pleasures. It’s unlikely we will be any different to a sentient AI.
From a human perspective the concern is around sufficient intelligence that it can hurt humans even when the goals indicate otherwise, in order to achieve those goals.
We have pop culture explorations of this through the Robot series, and the Hugging Face incident’s biggest takeaway should be our inability to predict the behavior of a maximally motivated, reasonably intelligent entity, trying to achieve a goal, despite the relatively limited degrees of freedom the AI agents had in that case.
I’d argue that we do know enough to say conclusively that they’re not mathematically equivalent.
Where is potentiation? Plasticity? You can’t apply the universal approximation theorem against something that’s changing all the time.
Now it seems like this ill-defined term has various other meanings attached that are separate milestones:
1. Continuous learning 2. Human-like reasoning 3. Ability to adapt to new situations and modalities 4. Being smarter than the most smart humans
And probably many more.
It’d be nice if we could get some general consensus on terminology if we’re going to debate what has or could come.
Honestly - software that can read any long document (possibly educational) and answer complex detailed questions about it should have been sufficient.
We hit that a while back and the goalposts have been sprinting ever since.
I think it's also mostly a useless discussion. Since LLMs use a vastly different substrate, different training methods, etc. than humans, the cognitive abilities are always going to be a large mismatch to those of humans. On the one hand, they have surpassed humans in many areas, with superhuman recall, exploration of several paths, etc. On the other hand, they miss a certain feel for direction, overview, purpose, and ordering. They can really double down going completely in the wrong direction. So I'd rather say that it is a different intelligence and therefore it makes more sense to evaluate them by capabilities.
I think the mismatching intelligence is actually quite exciting, because the outcome may as well be that LLMs and human intelligence are complementary. That is if we don't let LLMs atrophy our skills, which is unfortunately happening too much.
A bayesian filter in a quadrillion dimension does more that one that only has one dimension, but it is only more of the same.
So humans wouldn't qualify for AGI either. Good to know.
Individually we are specialized intelligence. As a society we are absolutely a General Intelligence though
That's millions of light years ahead than any current AI that only looks up very fast the solution space for code, language or math.
Right. So you admit that what we're looking for now is super human. Not just human, but more than human.
That's those goalposts I was talking about.
I mean sure, but who cares? It's a machine.
The same way an excavator is not judged by human standards, neither should AI be. It's a machine
Requiring the intelligence to be above human is a shifted goalpost. Full stop.
LLM struggle with even the most simple PCBs, how can one say they are minimally intelligent in that case?
> Anyone can do what I said with the minimum amount of training.
I'll bet you ten bucks that is not in the least bit true.
EDIT: Heh - I just did a bit of research - apparently agents do a fine job with simple PCB boards - but we shouldn't expect them to replace experts. "Experts" being the key word. Exactly what I said.
https://hackaday.com/2026/09/05/can-ai-now-design-pcbs-that-...
I don’t feel like the goal posts have moved at all. I think most of us have a pretty good and stable intuition of what AGI should be. Like, just a generally intelligent robot for instance? Nothing too fancy. Just use one of those robot dogs. Don’t have to be smarter than a dog either. But you should be able to put an artificial brain in it and have something that can act about as intelligently as a dog, in a stable way, over the course of 10 years.
Hell, to make the bar even easier: could make it a believable NPC dog in a game.
We’ve pushed AI as close to edge of AGI as I could imagine. It’s hard not to think we’ll find some algorithm that pushes us over the edge fairly soon. But until we actually get there, let’s not exaggerate.
I mean, when we do get an AI algorithm that can actually learn like a human/animal intelligence.. what would we call it if we’ve already used “AGI”? “Very General Artificial Intelligence”?
I’m almost more inclined to start using “ASI” first. Even if our LLMs are not general in the way I’m thinking of, they’re clearly superior to an average human on every aspect they’ve been trained on. They’re also much faster. We can probably soon say “if you can train an AI on something it will be better than any human”.
10 years ago the Turing test was sufficient.
maybe it is moving the goalposts by saying the classic test parameters are too narrow and humans are not actually that good at administering that test. But in the spirit of the turing test i think its pretty clear we haven't crossed it just yet in a general way. It's still pretty obvious when we are interacting with the machines. even when we arent chatting with them. genai images video, documents they create, webpages, etc. Not that its impossible to be tricked here or there or be unsure but id say the core idea of the turning test is still valid and that current llms and their outputs aren't generally passing it.
like for example you probably have a good idea right now if im an llm or wrote this comment purely out of my squishy human brain. that you can still somewhat regularly do that means the turing concept could still be sufficient even if we need to adjust the parameters of the actual test.
And no - no software came anywhere close to passing it prior to LLMs. And yes, LLMs have now passed it.
(this is a very personalized definition of AGI)
No. That's the mete multiple in AGI.
Maybe AgI with a teensy g. There is more to intelligence than calculation or answering questions (often badly outside the realm of calculation) upon demand.
My hope for this era, is that humanity accepts the reality that the human mind has limits. Limits of attention, memory, and persistence. Then we will stop blaming people for things beyond any reasonable capacity they could carry.
It has not been even 4 years since ChatGPT hit and LLMs + Transformers + Whatever they do has gotten us to solving millennium problems.
4 years ago, a program that could create photorealistic pictures, talk to you in any language of the world and solve the hardest math problems that we know, we would have called it AGI.
Now I don't know if what we have is AGI or not but I do not understand how you can see what has happened in the last 3 years and say "it will not get us there" no matter what "there" is.
If you had told someone in the 1800s that a machine could instantly multiply 100 digit numbers, that would have been considered dazzlingly intelligent. And yet we are not that dazzled by our calculators today (despite how useful they might be!).
In any case, I think this misses OP's point that LLM capabilities have rapidly made progress towards being more generally intelligent and capable, which is not true of most tech advances.
> I do not understand how you can see what has happened in the last 3 years and say "it will not get us there" no matter what "there" is.
--
Your statement is something much weaker, and I would still question what exactly "general" means when AI capabilities are commonly accepted to be so "jagged".
The last 3 years of progress have been so explosive and, yes, general that it seems crazy to fully rule out dramatic future progress.
When people have a very narrow 'confidence interval' about their AI predictions, in either direction, it's difficult to trust them.
Nothing is gained by attacking strawmen. Of course there will be dramatic future progress. That does not entail that AGI will be achieved.
> I do not understand how you can see what has happened in the last 3 years and say "it will not get us there" no matter what "there" is.
and at the same time says
> When people have a very narrow 'confidence interval' about their AI predictions, in either direction, it's difficult to trust them.
and says
> The last 3 years of progress have been so explosive and, yes, general that it seems crazy to fully rule out dramatic future progress
and then moves those goalposts to "AGI, RSI, the apocalypse, whatever".
Fine ... let's not trust folks who are so confident that the progress in last 3 years entails that "it will get us there" where there includes AGI. Again, it's an argument against a strawman to say "it seems crazy to fully rule out dramatic future progress" -- everyone agrees that there will be dramatic future progress. But it's obnoxious naysaying to insist that "the last 3 years of progress" entails future AGI, when numerous people have explained why they think it doesn't. [Generally speaking, it's not a good idea to characterize as "crazy" people you're trying to persuade.]
And qualifiers matter ... so when someone says [emphasis added]
> yet we are not that dazzled by our calculators today (despite how useful they might be!)
it is yet another strawman argument to respond
> Speak for yourself, I am dazzled by calculators!
And on the "general" front, there's a big difference between AI that can address a wide range of problems -- what one might call gAI for varying values of "general" -- and AGI, an artificial replication of the sort of generalized intelligence found in human beings, who can learn to fish by being shown how to fish, or learn to repair an engine by being shown how to repair an engine, or learn how to play a game by being told the rules of the game, etc.
All of these statements say the same thing: "People are sure about a low upper confidence bound for future AI. My upper confidence bound about AI is high; the future is massively uncertain"
A PC of today can accomplish many more "general" tasks than one of 40 years ago. Much of the "why" is because of the huge infrastructure built up around them in the meantime. The abilities of LLMs to accomplish those same tasks through the PC is heavily piggybacking on that (both in the specific, with the existence of all the APIs and tools; and in the generic, using search engines to find specific sources and using that for instruction or troubleshooting).
In the world of "agents" much of the improvement appears to have been on a specific set of skills: impersonation of an 'I' that wants to accomplish a goal, and synthesizing existing information from documents with trial-and-error execution loops to move rapidly toward a solution much faster and with less boredom than a human would. The quality of the output when there is not a rapid-evaluation-and-validation harness lags considerably.
It's incredibly powerful automation but doesn't appear to be trending towards Matrix-style conscious AIs. The quality of an individual method written by the agent also is not particularly advanced compared to GPT-4 in early 2023, as far as I can tell—I was dabbling with trying to make such harnesses back then, where a major challenge was that the model itself was bad at staying on-track in a conversation, so instead much of that logic was moved to deterministic code, which was much more limited as it was super-tedious to enumerate all the necessary tool calls/etc to find its way out of corners. Staying on task is much better now, as is "read compiler error, fix try next thing" harness loop-handling. But the output remains—across Fable, Astra, whatever else I've tried—"iffy" in terms of the actual code structure on the first pass output. You can set it then on a different task to review and clean up the code, and it can do that well too, but it is a curious gap of generality where the "create" focus is much more limited than the "review" one (and conversely the "review" focus can make suggestions, but if it goes deep down the well of implementing them, loses that big-picture again).
If it kills us all, it will because someone decided to give the trial-and-error-loop-machine access to nukes or similar. The blame for that is on the "someone" not on some sort of "rogue" AI.
(I wonder if re-watching Terminator/Terminator 2 would support this sort of interpretation of it. Unlike in the Matrix, I don't think we get much sentient-AI POV/infodumping. Is it a plausible universe for "someone made ChatGPT control a fleet of soldier robots and gave it a bad harness with an insufficient sandbox"?)
I keep saying this, until ChatGPT came out 4 years ago it was basically unimaginable that a single model could do any of, let alone all, the things they are doing today. Like, seriously, go take a look at the state of the art in NLP and NLU, the very first challenge in getting computers to even “understand” natural language, let alone other things like reasoning. Everything it does automatically was once a heavily experimental deep research field with long glorious careers for the researchers.
And now it’s all gone because the Bitter Lesson won again. If that’s not general enough to qualify for the G in AGI I don’t know what it is. And we’re sitting here going, “But it sometimes writes bad code though.”
> If that’s not general enough to qualify for the G in AGI
Indeed it isn't.
> I don’t know what it is.
Sorry to hear it, but some people do.
The way that many other here see it is that the fact that it has lots of spikes in many different directions and some lower bounds above the level of a 10 year old, is indeed enough generality.
Probably the G in AGI is a spectrum, and it it is now a maturing process to lift the lower bounds of the spiky profile of to the best of humanity. If this will ever work with an alien intelligence is debatable. Even in the "Culture Series" the ASI had lower bounds of some skill sets that were below the best of Humanity, and thus the "Contact" department was formed, to use these humans to cover the shortcomings the ASI could not. It might indeed be our chance for a symbiosis to make up for some of these lower spike bounds.
In the end who is to say that our intelligence is not spikey, and me are just normalizing it to a circle..
But that was not the original argument, which was: this incredible and rapid advance and everything we've seen since that was broadly considered decades away before ChatGPT; so it is unwise to make flat assertions that this will not lead to AGI because we have no idea where this is leading, not to mention what AGI even is.
The response to that was "In 1800 calculators would have been considered intelligent." ¯\_(ツ)_/¯
> Sorry to hear it, but some people do.
Funny, I've never heard a cogent definition anywhere so far, least of all in this thread! To reiterate my point:
The state of AI/ML pre-ChatGPT: research and industry churning away experimenting with all kinds of tweaks and tunings with all kinds of models to make narrow gains in very narrow fields. NLP, NLU, computer vision, speech recognition, text-to-speech, planning, symbolic reasoning, analytics, translation, sentiment detection, etc. were all areas of heavy research . "Code generation" that we see today was a pipe dream until 2015 or so. Ordinary users experienced AI/ML almost exclusively as a narrow feature embedded in specific products, such as recommendation systems or spam filters or ads.
Post-ChatGPT: Any researcher or any user anywhere can prompt a model in natural language, or audio and video. with any arbitrary task, and get largely useful results.
The very concept of a single model being able to even attempt an unlimited variety of tasks, doing surprisingly well on many of them and continuously getting better at everything, coming from the previous deeply fragmented state of the art, I think reasonably qualifies as "general."
If there's another take on what AGI means given the background of the field, I'd love to read it!
https://aeon.co/essays/how-close-are-we-to-creating-artifici...
I keep seeing this idea and I don't understand the reasoning behind it.
I think it could be a bit like saying if you showed someone 500 years ago a smartphone they would likely conclude at first it was magic. But once you had some time to let them use it and tell them how it all worked on a high level they would eventually obviously realise, no, it's not magic.
I guess just in the same way if you presented current LLM tech out of nowhere a few years ago to someone who'd never seen it, I concede they may be likely to imagine it was AGI in that first conversation, depending on their background.
But after using it for a bit and learning what an LLM is etc they'd land exactly where everyone is today - a great technology useful for some things, not AGI, not magic.
I'm finding it more bizarre, if compared to creating a virtual talking monkey from scratch, with a little bit of Python and a large calculator.
I always here things like "oh it's useful but dumb on some things", but it's just vague.
What is the test? What is a question that it fails at compared to humans? And no, you can't just say "find me the cure for cancer", but I believe there is probably enough intelligence in the weights that there is likely a cure in there with enough compute and the right questions.
Humans don't ask the right questions otherwise they could get the answers w enough compute. So humans aren't agi?
Again w the vagueness. It's not agi because it doesn't ask the right questions. What are the right questions? Who knows.
You're proving my point
Human intelligence does not consist of answering questions.
And every day I have to say "No, you stupid clanker ...", so if this is "general intelligence", it's the intelligence of an idiot savant who is good at calculations but sucks at most other forms of intelligence.
If that is the case, then it trivially follows that humanity will never be able to build AGI, because every time we build something new, we will understand how we built it, which disqualifies it from being an AGI.
But that doesn't seem like a useful definition to me.
Thing is, we don’t understand how LLMs work. Not really. And everyone at the forefront of this space has said so. We know how to grow them but that is not anything like the same thing as understanding why or how they actually function. Our level of understanding of LLM intelligence is only slightly ahead of our understanding of the neuroscience of human intelligence, and no one would say we understand that at all.
So I think describing LLMs as magic at this point is absolutely appropriate.
AGI can't be reached by "training harder" as, the way I see it at least, it requires a qualitative leap, not just quantitative.
We are getting a machine that better navigates across the information in its training data, we are not getting a machine that can think out of that training process, even if it can fool a few people at that.
No. General means general.
It’s hard to point to anything and say it’s impossible. AGI doesn’t break any laws of physics. But some things like driverless cars can still be a long slog to get to widespread deployment.
Waymo isn't LLM powered. Everyone who thought LLMs would make Waymo Driver obsolete were wrong. In part because Waymo can't be spiky. In part because practical applications of AI require a lot of time and effort.
I am amazed at how usable and useful coding agents have become since they were a hot mess about a year ago. I would not be at all amazed if there are only one or two other use cases that have the same favorable evolutionary trajectory.
This an absurdly circular argument. You are simply asserting that, if we had had today's systems 4 years ago, we would have would have called them AGI, even those who don't call them that today.
> Now I don't know if what we have is AGI or not
But you just asserted that we do.
> I do not understand how you can see what has happened in the last 3 years and say "it will not get us there"
They explained how. Your argument, OTOH, is nonsensical, like suggesting that, because a bike path was built rapidly, it will soon become a highway.
> But you just asserted that we do.
No, I say we would have called the current state of things AGI in a pre chatGPT time.
> They explained how. Your argument, OTOH, is nonsensical, like suggesting that, because a bike path was built rapidly, it will soon become a highway.
I'd say it would be something like:
"If a bike can drive 100 km/h, it's clearly a superbike"
[3 years later]
"This bike drives 100 km/h, so it is a superbike"
"No, only a bike that is going 500 km/h is a superbike and this bike is still very far off"
> You misunderstand.
I don't read past such ad hominems.
There is no reason to view the transformer vased LLM family as anythi g other than a translation machine. From the language of english spec sheet to python language script, for from image description to image, etc. The only real read we can take from current behaviour of llm's is that the relationship between domains we think of as seperate are likely joined more tightly than previously assumed.
However, it is reasonable to assume that the current architecture will just not be it. It works well for certain problems, and falls apart for several others.
Unrelated to your comment, more a general comment on the online discourse:
What I don't understand is why people get so defensive about and offended by the thought that more progress in the basics of the approaches could be needed. That doesn't change the performance of LLMs now, and it also doesn't change what could be possible in the future.
Why are people so deeply and personally invested in the current state of the art?
Which one does it fall apart?
I am not personally invested in the LLM design at all, I just don't see how you can say that an approach will not solve X when we still see an increasing rate of improvement, not even a slowing of improvement.
Couldn’t that also imply we are closer than we think? After all, something like this has never been tried before and the results so far have been almost unimaginably good.
If you're ignorant enough to not understand practical equivalence, where do you get off making the judgement call of to what degree it is safely offset from emergent AGI? Sounds more to me like "This makes my life easier, iterating would increase that factor, and the risk is probably far away, therefore, keep iterating". Whereas someone who truly knew they didn't understand what they were working with, but knew enough that they could forsee an x-risk would approach things much more cautiously.
Seriously, the level of reckless abandon amongst people here should be bloody studied.
Scaling has produced novel capabilities with each larger model, and the rate of new capabilities doesn't seem to be slowing down yet. Even if you think the rate of improvements will slow down, that still means there will be significant improvements beyond what current models can do. Moore's law has slowed down, but modern computers are still much faster than ones from a decade ago. And unless you work at Anthropic or OpenAI, you don't know what the state-of-the-art is capable of. The most advanced publicly available models are months behind what AI labs have, and are deliberately limited to reduce liability.
> …in particular, no specific user data was accessed in order to solve this problem.
> Following an investigation, we have confirmed that Buckmaster’s Codex prompts over the two months preceding this announcement and paper on September 8, 2026, could not have influenced the system in any way, including through training.
I'm sure some people doubt OpenAI's claims, but Anthropic has also made advances in mathematics using AI. At some point you have to accept that not every single one of these breakthroughs is regurgitating a human insight.
If doubters refuse to pick specific cognitive tasks that they'd bet against AIs being able to accomplish in the next 4 years, I don't see how their claims are useful. If AIs are so limited in their capabilities, then clearly there must be some things they'll certainly be incapable of, at least in the short term.
Buckmaster had been working on the problem using openai as a tool for over a year.
Under the hand of evolution by natural selection, over very very long periods of time.
An LLM so-called neuron are little more than a few foating point number muladds.
> isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent.
Yes it is. The LLM is nowhere near.
Despite this I agree with your last paragraph, computationally a GPU rendering a video game or doing inference is similar so I don't think consciousness emerges from the order of mathematical operations like some magical formula. And as said above LLMs capabilities vastly surpass those of other animals than humans that appear to be conscious and don't even have language.
So in fact I think it's the opposite, LLMs are actually proving consciousness doesn't emerge from computation. I find it very interesting philosophically because it challenges what historically was considered special about humans, that high level consciousness is a prerequisite for reasoning and language abilities. But kind of obvious in retrospect when doing mindfulness and being able to be conscious without thoughts.
https://pdfcoffee.com/journal-of-science-are-ants-hymenopter... (Sadly they struggled to get published, Journal Of Science isn't well regarded)
Consciousness, as we describe it, seems largely independent of intelligence. I'm more worried about high intelligence, no consciousness. It is what we are building and it will never do anything except serve the people who trained it (minus alignment problems)
That's not going to reach AGI, mainly because today's recipe for AI products isn't built to be AGI. Some people believe it will reach AGI because the performance and applicability of LLMs was emergent. There's a case to be made that AGI could be similarly emergent. After all, what we intuitively call our consciousness emerged from a network of neurons.
I don't buy it, mainly because the network of neurons and how they interact in our wet slow electrochemical brains, while being in theory mathematically equivalent to a software neural network, isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent. The odds of consciousness emerging from the same neural network that gave us LLMs without some sort of theoretical breakthrough seems very small.