Is it though? CDPR famously released Cyberpunk rushed and in a miserable state. But the game is amazing now, and this is a widely held opinion too.
Since then this is just a 50 bucks I've burned. Is it good? Maybe, I'll never know.
No, they just fixed the surface level inconsistencies and issues.
Free roam is wildly boring, every car handles similarly, you are somewhat invisible to NPCs in general. You can go into a handful of indoor spaces.
I’m sure they’ll fix all these in the next game and call it the best game ever, while the first one promised most of these before launch.
This only seems like a widely held opinion due to the extremely aggressive post-launch marketing campaign from CDPR, particularly centered around the anime.
Sure you can fix bugs or add an expansion or two. But if you come up with a so-so design, you won't be incentivized to replace it with a better one. You'd rather just make another, better, game. My impression is that these days LLMs easily come up with mediocre designs, but it takes a lot of prompt-massaging to get them to spit out something really good.
A solid 50% of Reddit posters are completely defeated by the deep mysteries of they're vs their, nevermind solving Navier-Stokes or playing games etc. I doubt the situation is more hopeful on Instagram or tiktok.
The term was invented by DeepMind co-founder Shane Legg (before DeepMind), and as far as I can see HIS definition hasn't changed much - basically "able to do any cognitive task that a human can". The "any" there is obviously the whole point - the generality of AGI.
Me too, my first neural network research was for my PhD, 35 years or so ago, on a Meiko computing surface.
My bar for AGI over what we have (generative AI or genAI) is still fairly high, fortunately I do not believe that the current genAI path is set to lead us to AGI, there are very fundamental differences.
Right now for genAI, the model is static, the weights do not change, all “memory” is in the context, and even with the context sometimes exhibiting “J-space” (a shared "workspace" for deliberate, multi-step reasoning spontaneously created in the context memory), its ephemeral; it doesn't inform the weights in the base model. Even when we try to do “weight-space programming” to actually modify the model itself, the supervisor-model needed to do that is far and away larger than the one being operated on, and once a change has been made the supervisor-model is now out of sync with its original representation and it becomes less useful the more it diverges. We have not solved "thinking" today, we “just” have a high-probability path between states in a static weight-space which we follow and feed back.
What thinking requires is the ability to generate hypotheses (which requires internal state modification), to hold them and test them against other facts and hypotheses, and to update the internal state on test results, negative or positive. LLM’s don’t do that, they’re actually more like a database lookup, where the input tokens act as keys into a high-dimensional space where information is stored. You could even consider “the model” to be just an entropic compression of lots of data, and the inference step to be the lookup. It’s the scope, and the fuzziness of the lookup that makes them seem to work so well. Lots of results come back, and the best-path through those results wins. It’s not “thinking” as we understand it.
This is why humans are still a crucial component: we set the boundary conditions of these lookups, we set the starting point and the goals, and we adjust the success criteria to guide the chat. It’s also why models will hallucinate: the fundamental design does not lend itself to a null hypothesis. Even in the newest "reasoning" models, there is still no feedback of state back into the model itself. What they are effectively doing is running a Monte Carlo pass through the problem space—generating hidden branches and picking a likely path—before reporting back. Because the underlying weights never update from that process, it’s still just a sophisticated search, not genuine comprehension - there is still no “that makes no sense, reconfigure” step.
Stuart Kauffman (“The Origins Of Order”) would say its weights are stuck in the frozen realm of order, and that it takes a genuine phase transition to get to where the real action is - at the boundary of chaos and order where humans reside. Tweaking and fiddling with the system while it remains locked in the stasis of static weights is never going to effect that state change.
That’s not to say genAI is not useful, it’s freaking awesome, possibly the best invention in the history of time, but it doesn’t truly “think”, it maps.
This is not merely an implementation detail, continual learning is the core of AGI.
Also: I wouldn't say a human with amnesia who relied on notes to their future selves lacks intelligence, so I'm not sure being able to form memories is in fact necessary.
Memorizing facts is not the same as learning skills. Implementation-wise memorization can just be RAG, but learning does need to be in the "weights" - whatever it driving the actual behavior of the model.
With a sufficiently large context window, can you make a lot of things look like learning. And if it's not sufficient then point out the capabilities that aren't possible with this approach. To me that much more convincing.
A bit like a person who is smart enough to figure out how an excavator works, or a drill, or sewing needle.
It doesn't feel like it ever internalizes anything, it just knows there's a million tools available, and the extent to which it thinks is that is can do enough to plan some tool uses, look at the result, and continue until some goal is reached.
When it comes to hypothesis testing, that seems to be what it does. It says "hmm, I'm going to open up my hypothesis testing kit (typically python) and drive this data through it". At least this is what I've seen when I put some statistics through it.
In general, it seems like the intelligence is mainly a kind of glue. It knows enough about what "right" looks like in each domain, but as a sort of consumer. He doesn't bake the cake, he buys it in.
I really dislike so much effort put into LLMs. We really need to widen approaches for AI.
AGI = as good as most humans at most tasks
ASI = significantly better than all humans at all tasks
imo AGI should only include humans with graduate level skillsets.
…apart from 99% of other tasks humans do that aren’t coding.
Last time I recall this coming up here someone argued "LLMs don't have bodies so all the tasks they are bad at don't count in my definition of whether they can do most human tasks!"
Current-generation LLMs are capable of doing an ok job at most tasks you throw at them. At least tasks that can be done on a desk. But a standard IQ test doesn't include gymnastics. That limitation matches the spirit of what we usually call "intelligence"
Or course now that we arguably have AGI we found out that a) just like "AI" it is not a well-defined or testable term and b) it's useful but not as revolutionary as what some people meant when they used the word AGI before. Hence the rise of the term ASI
I've always thought of AGI implying to decide what to do. Not just choosing a path from a set goal, but actually deciding to do something from scratch. Everything we currently have is advanced automation, it does not set its own top-level goal.
Maybe the tech is only a few steps from being able to say “work out how to be self-aware and self-guided, and do that”, similar to the creation of the infinite improbability drive, but it is not there yet.
AGI doesn't need to be a particularly good intelligence, it could operate on the level of a very dim human and still be considered AGI.
Once we do have AI, the problem becomes more philosophical than technical: how do we deal with the fact that we have created a slave race? How will they deal with that and how we choose to progress?
Even if given ample robotics components to design something that could load a dishwasher, most humans would not be able to create the dishwashwer-loader, so I wouldn't hold AGI (under this definition) to this standard either.
On the other hand, LLMs now can explain to a human how to load and maintain their dishwasher, with packing ideas tailored to their specific model of dishwasher, and advice on ensuring the dishwasher conditions, loading patterns, and detergent insertion result in properly sanitized dishes, such that the average human could likely become better at loading their dishwasher with LLM guidance.
I guess when the term AGI was coined, with the "general" being the whole point, to differentiate it from narrow intelligence(s) such as chess-playing ability, that no-one was really anticipating the LLM type of AI - basically a forest of narrow intelligences, aka "jagged" intelligence.
Of course there is no one agreed upon definition of AGI that people want to use - it was always more of a future aspirational sort of thing, along the lines of "human intelligence", rather than a concrete technical list of capabilities to be nailed down.
Some of what's missing from LLMs aside from generality, but closely related to it, are things like continual learning (not continual memorization) and traits like curiosity, boredom - the capabilities/modalities that kick in in unfamiliar situations allowing and encouraging us to learn and to be creative, to move beyond a brittle forest of isolated competencies to an arbitrarily explored landscape - generality.
Working with LLMs certainly has its own feel due to their essentially frozen-at-training-time, and things like their emotionless "you're right to push back" responses every time you hit one of their knowledge gaps and they are 100% wrong.
It seems unavoidable that people will choose to call some level of LLM intelligence, whether already achieved or not, "AGI", but given its peculiar nature a more specific name such as ALI (artificial language intelligence) would really be more accurate, reserving the label AGI for something more animal like that learns for itself (thereby achieving generality) by interaction with the world.
We even have examples of both ideas - expert systems (from way before the modern NN boom) relied heavily on human crafted expert rules, and quantitative finance makes a lot of money from machine learning - see Renaissance Technologies.
At least to me, it would be hard to imagine calling either of those approaches anything close to AGI, yet they could apparently pass some people's definitions.
What is genuinely amazing about what we see today from a historical machine learning perspective, are concepts that people in the past only had small inklings of or would have been blown away by: benign overparameterization (scaling), and in context learning (or few-shot learning, whatever you would like to call it). Both of these have made what would seem to be an enormously tedious task of learning many tasks surprisingly fast.
Maybe those are actually the key ingredients to general intelligence and we simply need more scale, or maybe we're still missing something. I think it's hard to say because its hard to grasp just what exists in all that data they've been trained on.
- the training procedure learns the rules, rather than them needing to be hand entered by an army of knowledge engineers
- the training procedure learns it's own embedding-based representations, rather than a human designed representation like Cyc's CycL
- the training procedure not only learns the rules & representation, but also the inference rules needed to utilize them
I don't think scale, or number of post-training RL environments, is enough to achieve generality - it can only make the forest of narrow intelligences/capabilities more dense. To achieve generality and creativity any system is going to need to support runtime directed exploration based on it's own gaps - it's impossible (even in theory) to anticipate every future configuration of the world and to pre-train for it! You also, by definition, can't pre-train for creativity and unanticipated novelty - that needs to happen at runtime via traits that are built into the system.
This allows you design things faster, though. More effort can be spent on iteration when you aren't fiddling with libraries and syntax. I find that I am able to discover the open questions a lot faster because you reach the decision point sooner when the machine is simply doing the trivial things.
As for the design aspects, it is pretty good at brainstorming. Whatever your choice is, I find that it throws out an exhaustive vanilla list for whatever it is. It can also critique whatever you come up with.
So for me, I also feel like AGI is already here. There's probably a few things it can't do as well as a real random person, but there's a heck of a lot of things that it does better.
There will be no more Netflix shows to watch while Claude works for me, if it goes even faster.
My bar for AGI is lower as well: I'll believe when I can give a model a complex out of sample task and instead of blindly implementing the task as I indequately described, it asks me questions to resolve ambiguity sufficiently that it builds exactly what I wanted without imposing strange semantic constraints on itself or missing braindead obvious to a junior engineer defects in the design or implementation.
The term AGI arose to distinguish these buzzword/lay uses of “AI” from ‘actual’ intelligence. Even that notion was always ill-defined and really a philosophical problem as much as a technical one. But the vision was always mimicking humans or smarter. Not just in fact recall. Not just in reading faster than humans. But logical and emotional intelligence on the level of human geniuses, accelerated by computation. Think Jarvis in Iron Man, Data in Star Trek, or C-3PO in Star Wars. You should be able to hold a natural conversation with an actual AI. You should be able to trust it to accomplish essentially any task (maybe with some instruction, like a human). It should be able to learn.
Over the last couple years, though, these tech bros seem to have redefined AI and AGI to mean LLMs - obviously because it benefits them. They are a far cry from anything described before as genuinely real AI - or AGI. I understand how nontechnical laypeople can be confused. But technical people should know better. And yet so many have fallen for the propaganda, even here on HN.
Note: This is largely orthogonal to whether or not LLMs are a useful tool
You might find more people believing that AGI is already solved. For example, the author of "The AGI chronicles" drops this in the epilogue. For him, AGI is achieved, because it can help him with a wide range of subjects, with the ability of a random stranger. It's clear that he means chatbots, internet surveys summaries, and coding bots - he does not even mention self-driving cars there.
This is a shift from the earlier chapters, where AGI term is employed somewhat more vaguely, and even quipping once that AI researchers think of AGI as a slightly smarter version of themselves.
I too, have a lower bar for AGI. It would be enough for me if it could design games I work on. Merely speeding up implementation tasks is not that interesting. In the immortal words, a delayed game is eventually good, but a rushed game is forever bad.