Being in the tail end of my PhD, I want to second this sentiment. I'm not even bullish on AGI (more specifically HLI) in 50 years. Scale will only take you so far and we have to move past frequentism. Hell, causality research still isn't that popular but is quite important for intelligence.
I think people (especially tech enthusiasts) are getting caught in the gullibility gap.
Causal understanding is going to be a bit difficult to prove tbh.
I'm not sure why you think this falsifies intelligence. There are plenty of puzzles and illusions that trick humans. The mere presence of conceptual error is no disproof of intelligence, any more than the fact that most humans get the Monty Hall problem wrong is.
And in general, if you're going to be condescending, you should actually make the counter argument. You might make fewer reasoning errors that way.
The point isn't that we can or cannot extrapolate. The point is that it is worth investigating to see if the trend line holds up.
I think it's that coupled with inserting their own often fanciful projections to fill the void and gaps between the lack of understanding how the mechanics works. Here is a good example of influencers laboring under this narrative [0] and a glimpse as to why it's so effective and pervasive.
I'm conflicted, because I think this is precisely what keeps many Corps and VCs throwing money into this tech thinking it's just right around the corner, as, I'm also studying AI and ML, but I also realize how implausible AGI is in my lifetime given where we are and how narrow parameters these algos function in to yield some degree of accuracy and more specifically how many errors there are to do so as it's just part of the process--learning how gradient descent works was pretty eye opening.
The Marketing has made many people not just fall into a gullibility gap as they make it seem like some sort of black magic, when it's really just statistics fed with immense amounts of specific data, but influncers make it seem like it's capable of ending the World (Elon) when in reality people should be more worried about big tech's capability at aggregating data points from info people willfully post online to exclude you or cluster you into a certain group because of your use of free apps and selling it the highest bidder which are tied to your real ID. Or the algos with weights (biases) that are used and bundled into outsourced/automated HR solutions.
That doesn't really qualify you to make these anti-predictions any more than tens of thousands of other people working in this field. There's also this phenomenon where experts close to a problem are worse at predicting the future because they are so intimately familiar with the problems that they can't see the forest for the trees.
To me there is one major signal that AGI is much closer than 50 years away and that is the breakthrough with Transformers. Now we have a general purpose architecture for predicting the future that works really well, which is a key component in the brain (cortical columns). Clearly we're still at the early stages, but 50 years is a loooong time. 50 years ago we didn't even have the Altair 8800.
Look at the work result of folks like engineers at Google - state of the art in AI and driving most of the trends in AGI. Academic opinions are shot down quite quickly there, PhD hiring is mostly frowned on, even in teams working on AGI related tech.
It might be expected that PhD experience provides surprising little insight here unfortunately. Academic smugness is not something that should be offered as part of an opinion- just like many computer science related concepts, progress moves so quickly in these topics that by the time academics reach and start to iterate on concepts, companies have already moved very far beyond.
This isn't intended to be condescending to academics, just real life experience I've had working closely with both crowds and seeing this firsthand. It's a very palpable thing in industry, an Academic gives an opinion on AGI and engineers roll their eyes - since Academic perspectives are currently so useless or out of date for any current, meaningful progress.
It feels a bit like a lot of these fringe science things (electromagnetic fields causing harm etc) where people's argument is that nobody's shown it isn't true. Well if it's true, propose a mechanism and study it. You're not really doing research if you just make up a conclusion and use the fact that it can't be definitively refuted as evidence for it
"What are the remaining unknowns that would need to be solved to do X and why are they difficult?" is valid question to ask experts when trying to further your own understanding.
Philosophers, psychologists, neuroscientists, and many others have been contemplating the definition of intelligence for centuries and we're still not quite there. But we do have some good ideas. One of the important aspects is the ability to draw causal relationships. This is something you'll find state of the art (SOTA) text to image (T2I) generators fail at. The problem is that these models learn through a frequency based relationship with the data. While we can often get the desired outcome through significant prompt engineering it still demonstrates a lack of the algorithm really understanding language. The famous example is "horse riding an astronaut." Without prompt engineering the T2I models will always produce an astronaut riding a horse. This is likely due to the frequency of that relationship (people are always on horses and not the other way around).
Of course, there are others in the field that argue that the algorithms can learn this through enough data, model scale, and the right architecture. But this also isn't how humans learn and so we do know that if this does create intelligence it might look very different ("If a lion could talk I wouldn't be able to understand it"). Obviously I'm in the other camp, but let's recognize that this camp does exist (we have no facts, so there's opinions). I think Chomsky makes a good point: our large language models have not taught us anything meaningful about language. Gary Marcus might be another person to look into as he writes a lot on his blogs and there's a podcast I like called Machine Learning Street Talk (they interview both Marcus and Chomsky but also interview people that have the opposite opinion).
So another thing, is that machines haven't shown really good generalization, abstraction, or really inference. You can tell a human a pretty fictitious scenario and we're really good at producing those images. Think of all the fake scenarios you go through while in the shower or waiting to fall asleep. This is an important aspect of human intelligence as it helps train us. But also, we have a lot of instincts and a lot of knowledge embedded in our genetics (as does every other animal). So we're learning a lot more than what we can view within our limited lifetimes (of course our intelligence also has problems). Maybe a good example of abstraction is "what is a chair." There's a whole discussion about embodiment (having a body) and how language like this may not be able to be captured in the abstract manner without a body since a chair is essentially anything that you can sit on.
I feel like I've ranted and may have written a mess of a lot of different things haha. But I hope this helps. I'll try to answer more and I'm sure others will lay out their opinions. Hopefully we can see a good breadth of opinions and discussion here. Honestly, no one knows the answers.
Not all us do that.
A chair is both what I see as well as what I perceive. And it is neither. But, I never build a visual model of it when my eyes aren’t seeing it.
> Philosophers, psychologists, neuroscientists, and many others have been contemplating the definition of intelligence for centuries and we're still not -quite there
Given the decade-old, seminal, widely cited papers by Legg (co-founder of Deepmind) & Hutter (long-term professor of ANU with multiple awards, currently researcher at Deepmind) titled "Universal Intelligence: A Definition of Machine Intelligence"[1] and "A Collection of Definitions of Intelligence"[2] and the whole rigorous mathematical theory of universal bayesian reinforcement learning aka "AIXI"[3] developed during the last decades which you fail to mention, this soft language of "not quite there" looks evasive, bordering on disingenious even.
Citing Chomsky, with his theorems which were interesting at their own time yet quickly faded into irrelevance once the modern assumptions of learning theory were developed, and Marcus whose simplistic prompts are a laughing stock among degreed and amateur practicing researchers alike brings little of value to the already biased discussion.
> Of course, there are others in the field that argue that the algorithms can learn this through enough data
Such as Rich Sutton, one of the fathers of Reinforcement Learning as an academic discipline, whose recent essay "Bitter Lesson"[4] took the field by word of mouth, if nothing else. It takes a certain amount of honesty and resolution to acknowledge that one's competitive pursuit of knowledge, status and fame via participating in academic "publish or perish" culture does not bring humanity forward along a chosen axis, compared to more mundane engineering-centric research, many people commend him for that.
> So another thing, is that machines haven't shown really good generalization, abstraction, or really inference.
To play a devil's advocate: "Scaling is all you need" and "Attention is all you need" are devilishly simple hypotheses which seem to work again and again when we reluctantly dole out more compute to push them further. With a combination of BIG-bench[5] and an assortment of RL environments as the latest measuring stick, why should we, as a society, avoid doling out enough compute to take it by storm in favor of a certain hypothesis implicit to your writing, which could as well be named "Diverse academic AI funding is all you need"? Especially when variations of this hypothesis have failed us for decades starting with the later part of the XX century, at least if we consider AI research.
> but let's recognize that this camp does exist (we have no facts, so there's opinions).
With all due respect, the continuing lavish (by global standards) funding of your career and careers of many other unlucky researchers should not be a priority when such massive boon to all of humanity as a practical near-HLI/AGI is at stake. "Think of the children", "think of the ill", think of everybody who is not yourself - I say as a taxpayer, hoping for AGI checkpoint to be produced by academics and not the megacorporations, and owned by the people, and not the select few.
Which - the proprietary nature of FAANG-developed AI - might be a real problem we are going to face this decade which is going to overshadow any career anxiety you may have experienced so far.
1. https://arxiv.org/abs/0712.3329
2. https://arxiv.org/abs/0706.3639
3. https://arxiv.org/abs/cs/0004001
4. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Search turns up very little except a book by a Christian author. Could you define what you mean by gullibility gap?
what are the esteemed epistemological frameworks in the AGI space? from the outside looking in, i see CNNs and lots of correlation-based approaches: it looks like the field settled on consequentialism. is that not the case?
When I read these kind of threads, I believe it's "enthusiast" laypeople who follow the headlines but don't actually have a deep understanding of the tech.
Of course there are the promoters who are raising money and need to frame each advance in the most optimistic light. I don't see anything wrong with that, it just means that there will be a group of techie but not research literate folks who almost necessarily become the promoters and talk about how such and such headline means that a big advance is right around the corner. That is what I believe we're seeing here.
As far as I am imagining it, current models are pipelines of various trained networks (and more traditional filters in the mix) that operate like request-reply. Why can’t you just connect few different pipelines in a loop/graph and make an autonomous self-feeding entity? By different I mean not looping gpt to itself, but different like object detection from camera vs emotion from a picture based on some training data. Is it because you don’t have data what is scary or not, or for a completely different reason?
Probably not. I did attempt to give a high level explanation in another comment but I think there is this naive belief that complex problems can be distilled into terms that laymen can understand. This is such a complex problem that is so ill-defined that experts argue. I'm not sure there's really a good "explain it like I'm an undergrad who's done ML courses" explanation that can be concisely summed up in a HN comment.
Naive people like Richard Feynman, who said if you can't explain an idea to an 8 year old, you don't understand it? Can you tell us why you think Nobel Prize winner Richard Feynman is naive?
> Hell, if I could explain it to the average person, it wouldn't have been worth the Nobel prize.
> I can't explain [magnetic] attraction in terms of anything else that's familiar to you. For example, if I said the magnets attract like as if they were connected by rubber bands, I would be cheating you. Because they're not connected by rubber bands … and if you were curious enough, you'd ask me why rubber bands tend to pull back together again, and I would end up explaining that in terms of electrical forces, which are the very things that I'm trying to use the rubber bands to explain, so I have cheated very badly, you see."
> we have this terrible struggle to try to explain things to people who have no reason to want to know. But if they want to defend their own point of view, they will have to learn what yours is a little bit. So I suggest, maybe correctly and perhaps wrongly, that we are too polite.
My best guess is that this misattribution comes from quote ABOUT Feynman and a misunderstanding at what is being conveyed.
> Once I asked [Feynman] to explain to me, so that I can understand it, why spin-1/2 particles obey Fermi-Dirac statistics. Gauging his audience perfectly, he said, "I'll prepare a freshman lecture on it." But a few days later he came to me and said: "You know, I couldn't do it. I couldn't reduce it to the freshman level. That means we really don't understand it." - David L. Goodstein
Which doesn't mean what you're using your "quote" to mean. As I stated before, we (the scientific community), don't even know what intelligence is. We definitely don't understand it so I'm not sure how you'd expect us to explain it to an 8 year old. Lots of things can't be explained to an 8 year old. Good luck teaching them Lie Algebras or Gauge Theory. You can have an excellent understanding of these advanced topics and the only way that 8 year old is going to understand it is if they are a genius prodigy and well beyond a layman. This quote is just illogical and only used by people who think the world is far simpler than it is and are too lazy to actually pursue its beauty. They only want to sound smart and they only will to those dumber than them.
Stop saying this and get off your high horse and hit the books instead.
Maybe Feynman was right, we're being too polite. There are a bunch of people in this thread that are not arguing in good faith and pretending to be smarter than people that are experts and performing mental gymnastics to prove that (you are one but not alone). If an expert is telling you that you are using words wrong, then you probably are. Don't just assume you're smarter than an expert. You don't have the experience to have that ego.
https://en.wikiquote.org/wiki/Richard_Feynman
https://www.sciencealert.com/watch-richard-feynman-on-why-he....
Then any discussions will be highly handicapped by the fact most still view human intelligence as something special. The field is still very much awaiting its urea synthesis moment.
The sperm whale has a brain that is several times larger than ours.
What do you do differently in your AGI design to get a human, gorilla, or whale brain?
If you have an idea for a technical approach then go ahead and build it. See what happens.
https://twitter.com/NandoDF/status/1525397036325019649
Someone’s opinion article. My opinion: It’s all about scale now! The Game is Over! It’s about making these models bigger, safer, compute efficient, faster at sampling, smarter memory, more modalities, INNOVATIVE DATA, on/offline, … 1/N
Solving these scaling challenges is what will deliver AGI. Research focused on these problems, eg S4 for greater memory, is needed. Philosophy about symbols isn’t. Symbols are tools in the world and big nets have no issue creating them and manipulating them 2/n
For me the key breakthrough has been seeing how large transformers trained with big datasets have shown incredible performance in completely different data modalities (text, image, and probably soon others too).
This was absolutely not expected by most researchers 5 years ago.
But I think a big part you're ignoring here is how training has changed in the last 5 years. You look back at something like VGG where we go depthwise with CNNs vs something like Swin and there's a ton more complexity in the latter (and difficulties in reproduction of results likely due to this). There's vastly more uses of data augmentation, different types of normalization (see ConvNext), better learning rate schedules, and massive compute allows for a much better hyper-parameter search (I do also want to note that there are problems in this area as many are HP searching over the test set and not a validation or training set).
Yeah, the field has come a long way and I fully expect it to continue to grow like crazy but AGI/HLI is a vastly more complex problem that we have a lot of evidence that the current methods won't get us there.
A suppose something approaching AGI, an intelligent agent which can act on a _wide_ variety of input channels and synthesize these inputs into a general model of the world, could use channel-specific approaches / stages before the huge "integrating" transformer step.
why always transformer step?
I think the main three obstacles in our way are imbuing models with (from least to most difficult):
1. continuous execution, which would be necessary for a real-time "stream of consciousness"
2. discreet short and long term memory and the ability to use it productively
3. plasticity at runtime
1 is just a matter of finding more efficient architectures and cheapening compute, both fronts we're constantly making progress on. 2 is a little more muddy, but there are already some really impressive models that have retrieval mechanisms which allow them to search through databases to help them contextualize prompts which I believe is a step in the right direction. 3 is probably the hardest in my opinion--I think we need several fundamental breakthrough discoveries before our models are anywhere near as plastic as, say, a human brain. My hypothesis is that exceptional plasticity is necessary for sentience, but obviously that's just my own personal opinion.
Anyways, thought I'd chime in with my two cents. I'd love to hear others' thoughts on the feasibility/infeasibility of near-term AGI.
I would define sentience as an awareness of oneself in relation to one's environment, that generalizes to different aspects of self and environment (eg. social, physical etc)
One of the key aspects of intelligence is learning from 3rd-person perspectives - eg. you observe another ape hunting with a stick, and decide to emulate that behavior for yourself. This seemingly simple behavior requires a lot of work, you have to recognize other apes as agents similar to yourself, then map their actions to your own body, then perform those actions and see if it had the intended outcome.
Current RL agents are not capable of this. The data used for GATO and similar agents for imitation learning are in first person perspective. If an agent could learn from a 3rd person perspective, via direct observations of its environment, I would say that would be the beginning of machine sentience.
10 years ago no one was talking about AGI, researchers didn't even call it "AI" due to its association with the previous AI winter. There were certainly advances in ML but I think if you asked anyone whether those approaches would lead to real AI it would be an unequivocal no.
It also seems like unless something goes very very wrong with the exponential growth curve in computational capacity humanity has had, we'll be able to straight up simulate a human brain (probably well before) 2100.
An example: In the Chinchilla paper [1], the authors suggest that most big transformer models are undertrained, and that we will probably see diminishing returns in scaling up the size of networks if we don't also scale up the size of the datasets. They have a subanalysis where they extrapolate out how big datasets will need to be for larger models. If you believe it, then within two to three generations of big NLG models, we may need on the order of 100 trillion text tokens. But it's not clear if that much text even exists.
In the RL space, a sufficiently complex, stochastic environment is effectively a data generator.
I haven't heard of any groups who are studying data constrained learning in the context of LLMs, but that will probably change as models get bigger. And at that point, architectures with better scaling laws may be right around the corner, or they may not. That's the pain of trying to project these things into the future.
RL is such a different field that you can't apply these scaling laws directly. eg. agents playing tictactoe and checkers would stop scaling at a very low ceiling.
Who is we exactly? As someone working in AI research I know no one that would agree with this statement, so im quite puzzled by that statement.