I still see this as an open question. You’re certainly correct that this kind of AI research is seriously out of vogue, but it seems to me that while “modern” brute force compute AI puts up impressive results and is a hugely useful technique, it has made exactly zero progress on anything that could be conceived of as “general intelligence”. It just doesn’t seem to me to be a thing AI researchers are even interested in any more. Like, the Twitter program that uses AI to crop images based on a dataset of a gazillion cropped images is pretty far from Turing’s thinking machines.
I don’t know the way there, but it always seemed to me that the old-style AI research in the GEB style is still a rich vein we haven’t come close to mining out.
At least we can train language models in hours or days rather than years. If you view it that way we've made quite a bit of progress!
Your second point doesn't make sense though. Language models don't understand language in the way that humans do. They just maximize the probability of the next word given some corpus. This is completely different than what humans do.
The important point to observe here is that what a kid does is fundamentally different than what GPT-3 does:
- GPT-3 learns the word that minimizes perplexity given some context
- A kid learns the word that helps them accomplish some task in some environment.
In the learning process, children get feedback from their environment - including the responses of other agents in the environment (eg Mom, Dad, friends, etc). This is going to be far more memory-intensive than some text files. Also important to observe is that the child's experience can't be reduced to the audio, video and haptic streams they get, because the audio and video stream depend on their own actions. So you need the conditional statements that were embedded in the environment the kid was learning in. All of which is to say this is a little heavier than ascii.
Edit: From your other comments, it seems we agree that bigger neural networks with more training data are not going to yield some quantum leap where GPT-3 starts talking. I'm not at all suggesting that throwing more data at the existing architectures is a fruitful direction for understanding cognition. (In general I think NLP is utterly pointless as regards understanding cognition, and if that's your goal you should focus on vision and reinforcement learning.) But my point is that we can't expect that a smarter architecture on a smaller corpus will get us anywhere. If we ever develop machines that are intelligent in some robust sense of the word, their "training data" will most likely be a physical environment with other agents in it.
I never said we understand the right architectures / algorithms that are necessary for robust machine intelligence. In fact I explicitly said that we don't (which answers your straw-man question).
https://www.theatlantic.com/magazine/archive/2013/11/the-man...
Look at the other claims in this article from 2013 for comparison:
Consider that computers today still have trouble recognizing a handwritten A.... In Hofstadter’s mind, there is nothing to be surprised about. To know what all A’s have in common would be, he argued in a 1982 essay, to “understand the fluid nature of mental categories.” And that, he says, is the core of human intelligence.
I would say the modern era of AI was kicked off with AlexNet in 2012 and hit its stride a few years later. So, I believe this quote and this article are really referring to pre-GPU AI techniques.
Basically, the predictions from this era just underestimated the value of scaling the data. Modern AI has several impressive achievements. It can certainly recognize an A. Image recognition and voice recognition are now critical parts of real products that help people in their everyday lives.
At the same time, it's true that current AI techniques might not get us all the way to AGI. We'll just have to wait and see. But I think it's important to recognize that we have had real progress in the modern AI era that has seriously outperformed the pessimistic expectations from ~2010.
Hofstadter identifies novelty typefaces where A has none of the characteristics that you might associate with the letter. Often A is not a triangle, sure, but sometimes it has no upper bar, no left stroke, no right stroke. Sometimes the bars are defined by a transition from negative to positive space.
Sometimes A is made of 20 strokes, or a pattern of dots, or an open eye where the half-circular upper lid defines the triangle and the pupil is the bar. A neural network trained on conventional As would not conclude that any of these are an A.
These shapes wouldn't necessarily read as A in isolation. Sometimes they only read correctly when you're looking at text using a whole typeface defined on the same principles.
Modern AI still cannot recognize that kind of A even though a human easily can. There are ways to make an A that require a human to understand an idea of "letterness" that's different than anything they've seen before.
I'd recommend reading the original essay. The illustrations make this clear in a way my description can't.
Feels like we found a magic bean
I would think this is a consequence of it being a very hard problem. ML gets all the industry funding and publicity because you get results that are immediately useful.
You can work on General AI for decades and come up empty handed since there doesn't seem to be an incremental approach where intermediate results are useful. So it's closer to esoteric mathematics or philosophy in terms of "being en vogue".
So I see this mostly as a reflection of the academic landscape in general. Funding is more focused on application and less on theoretical / fundamental research.
I think that it's possible that 95% of the economically obvious value of AI will be in the not-real kind, and that it will be captured by applied statistics and other "mere tricks." It could be a long, slow march of automating routine jobs without ever directly addressing Turing's imitation game. And since most of the obvious labor replacement will have already been done that way there may be fewer resources put into chasing the last 5% of creative, novel, non-repetitive work.
But I suppose that’s not as exciting as a rogue AI.
The actually concerning aspects about AI safety in my opinion are more about misuse/abuse as well as critical processes/decisions without human oversight. To be deliberately malicious, AI has to be strong, if not general. Both are very difficult. It is good to think about those things though
Perhaps you will enjoy this: https://heurolabs.atlassian.net/wiki/spaces/AIS/overview
If we start using words this way, you could say that a deterministic fibonacci function "learns" the value of fib(5) by computing it. "Machine learning" becomes synonymous with "computer science."
ML means that the function that maps inputs to outputs is learned from data. This definition is overly restrictive (but match e.g the behavior of neural networks).
Wikipedia has a more inclusive and useful definition of what qualifies as ML: Machine learning (ML) is the study of computer algorithms that improve automatically through experience and by the use of data.[1] So a ML algorithm allow to automatically improve throughput at a tasks through learning the representation of the data it is fed. Considering this definition, we can realize that the most used ML algorithm in the world is Pagerank (for improving the ranking of Google search results). And surprisingly, this algorithm is non-probabilistic. It attribute higher weight proportionally to URLs that are the most linked to by other sites. I.e it basically learn the structure of the graph that is the web. And its performance is data driven. So it's not just about causally memorizing the structure of a graph but also about reusing its past memories for newer queries, then we can effectively talk about machine learning. Pagerank diagram -> https://en.m.wikipedia.org/wiki/PageRank#/media/File:PageRan...
I'll give you an example of a non statistical machine learning algorithm that I am currently developing: Semantic parsing is the task of encoding semantic meaning in a graph from a natural language text input.
This graph can then be used for semantic question answering.
It is rule based and it will learn the semantic structure of the text. The more data you fed it the more knowledge it can encode. Hence it's performance at question answering increase with experience. It is causal and yet it fit the useful definition of machine learning cited above.
I'm currently reading Surfaces and Essences from the same author, and it's so far been most illuminating. He very convincingly presents the thesis of the analogy being the foundation of each nd every concept and human thought. If someone could manage to apply and translate those insights into real algorithms and data structures to play with, that would be IMHO a big step towards general AI, or the very least a much more human like AI approach with its particular applications.
Analogy to me comes after you have stabilized "shapes" where you can draw comparison between a new input a set of stable patterns at which point you can point out the analogy. Syllogism may be closely related/involved.
I am not disputing that being able to draw parallels and employ analogies are related to intelligence. I am just not so sure that this is how you can synthesize intelligence. It is all very complex of course, and there is a school of thought that cognition is very closely related to sensory and embodiment (i.e.: embodied cognition ).
I think Aubrey de grey switched from AI to longevity because he figured he will need a long life to understand intelligence :)
Personally, I am still working on it but had adjusted the scope a bit.
GPT-3 is hilarious. If that's not a sign of our inching towards general AI, nothing is.
From https://www.gwern.net/GPT-3#devils-dictionary-of-science
(The "Navy Seal Copypasta" section is even funnier, but it's not really HN-appropriate.)
--
“A role for…” [phrase]
A frequent phrase found in submitted and published papers; it often indicates that the authors have nothing to say about the topic of their paper. In its more emphatic form, “A role for…” usually indicates a struggle by the authors to take a side on an issue, after a lengthy attempt to be both non-committal and a supporting party to all sides, as often happens in “molecular and cellular” or “basic and translational” research.
“Reviewer” [noun]
A participant in the review of a grant, paper, or grant proposal. In spite of being in a poor position to assess the merits of a proposal, reviewer tends to demand that authors submit their data for statistical analysis and back their results with it, which the reviewer usually does not. Reviewer usually requires that the author cite his or her own work to prove that he or she is worth reviewing. It is also assumed that the reviewer can detect the slightest amount of bias in any paper, which the reviewer also assumes has not been corrected for."
“Rigor”
Something for scientists to aspire to, a state of mind that would not be required if scientists could be trusted to do their job.
“Science”
A complex web of data, opinions, lies, and errors, now considered the most important (because most expensive) technology in the modern society. To remind you of this, you will frequently see scientists and editors use the word, claim to do something for the sake of science, or see it used as an adjective.
“The topic of the paper”
A wide-ranging category of things or ideas that may not have been relevant when the paper was written, but which the authors believe the paper should be about. Often, the topic is too broad or a non-topic, but is occasionally useful in order to generate support for yet another set of related papers, conferences, seminars, webinars, and so forth, which in turn are used to generate more data for “new findings”, which, after they are manipulated enough, may end up being published and generating yet more data to support a “re-review” of the original paper or other things.
“Validation step”
Another name for a random setting of a parameter of a model, simulation, or algorithm.
“Writeup”
A form of scientific communication in which the author states the information he or she wanted the readers to extract from the paper while making it as difficult as possible for them to find it.
“Writer’s block”
A common affliction among students, arising from various causes, such as: their desire to sell their ideas for a profit, their inability to realize this desire, the fact that their ideas are not selling and will not be bought, and the delusion that most of the wealth and fame in the world would be theirs if they would spend enough years doing science.
You might be interested in Jean Piaget's work on the development of intelligence, and his observation on children development and various stages of intelligence.
BTW what is general intelligence in your opinion?
So instead of believing GEB+IAASL haven't stood the test of time, I prefer to believe that current technology is akin to banging raw power together in hopes of seeing a spark of cool in the minor league of "this AI does this nifty (and maybe useful) thing better than humans" but we haven't yet upgraded to major league of AI choosing their own goals and what ever may emerge from that.
(It may be that I'm due for a re-read!)
Just to play Devil's Advocate ever so slightly: there are people out there who would say that there hasn't been any "progress in AI" for quite some time, or at least very little so. And they would probably argue further that the apparent progress you are referring to is just progress in "mere pattern recognition" or something like that.
I'm not sure myself. I do lean at least a little towards the idea that there is a qualitative difference between most of modern ML and many aspects of what we would call "intelligence". As such, my interests in all of this remain around the intersection of ML and GOFAI, and the possibility of hybrid systems that use elements of both.
But I can't rule out the possibility that it will eventually be found to be the case that all of "intelligence" does indeed reduce to "mere pattern recognition" in some sense. And Geoffrey Hinton may be correct in saying that we don't need, and never will need, any kind of "hybridization" and that neural networks can do it all.
A neural network that just "feels" that Fermat's Last Theorem is true, is much less intelligent than one that can produce the proof of it and present that to us, so we can trust that what it's saying is true.
If you can't do symbolic manipulation, you are not really intelligent, artificial or otherwise, I would say.
It doesn't seem completely off. What the author is describing—and what we're seeing—is increasingly sophisticated algorithms, that are getting better and better at answering more and more narrowly defined questions: https://www.theguardian.com/technology/2021/mar/08/typograph...
It is the age old "rote memorization" versus "learning". In large, I suspect those do not have a clear line between them. Such that emergent behaviors are expected and can be taught.
Do let me know if I misrepresented it. I thought it was a clever way to show that the models are not finding inherent relations in the data, by just randomly labeling all training data and having the same speed/accuracy on the random labels as on the real ones.
Edit: I worded the above poorly. They showed the models were capable of merely learning the training data. Despite that, they also showed that moving the trained models from test to validation did not have a linear relationship with the amount of noise they added to the data. That is, the training methods seem indistinguishable from memorization, but also seem to learn some generalisations.
Maybe it's a bit of both. Sure, large DL models use lots of compute, but successful DL applications require some insight into the problem. For some reason, people like to de-emphasize this insight. The story is that a DL model will discover by itself which features are important, which are not, and you just provide the training data, and press a button. Thousands of people do just that, and end up with mediocre results. Thousands and thousands of absolutely mediocre papers get published, and receive acclaim instead of derision.
The truly boundary shifting results always use deep insight. Like what comes out of DeepMind (AlphaGo, AlphaZero, AlphaFold).
As has been argued in these pages many times before there has been no obvious progress in AGI, although certainly progress in AI in its weak sense has been impressive in the last few years.
I know you didn't say AGI but its important to make that distinction as the book was very interested in that as a subject.
Intelligent Systems will be developed and deployed in this decade. However, full equivalence to humans will be probably not be achieved in this decade.
See the following for more information:
"Robust Inference for Universal Intelligent Systems" https://papers.ssrn.com/abstract=3789701
[Gödel 1931] seemed to have settled the issue of inferential undecidablity{∃[Ψ:Proposition](⊬Ψ)∧(⊬¬Ψ)} in the positive using the proposition I'mUnprovable, such that I’mUnprovable⇔⊬I’mUnprovable.
However, existence of I’mUnprovable would enable the following cyberattack [cf. Wittgenstein 1937]:
Proof of a contradiction in foundations: First prove
I’mUnprovable using proof by contradictions as follows:
In order to obtain a contradiction, hypothesize
¬I’mUnprovable. Therefore ⊢I’mUnprovable
(using I’mUnprovable⇔⊬I’mUnprovable).
Consequently, ⊢⊢I’mUnprovable using
ByProvabilityOfProofs {⊢∀[Ψ:Proposition<i>](⊢Ψ)⇒⊢⊢Ψ}.
However, ⊢¬I’mUnprovable (using
I’mUnprovable⇔⊬I’mUnprovable), which is the
desired contradiction.
Using proof by contradiction, ⊢I’mUnprovable meaning
⊢⊢I’mUnprovable using ByProvabilityOfProofs. However,
⊢¬I’mUnprovable (using I’mUnprovable⇔⊬I’mUnprovable),
which is a contradiction in foundations.
Strong types prevent construction of I’mUnprovable using the following recursive definition: I’mUnprovable:Proposition<i>≡⊬I’mUnprovable. Note that (⊬I’mUnprovable):Proposition<i+1> because I’mUnprovable is a propositional variable in the right hand side of the definition of I’mUnprovable:Proposition<i>. Consequently, I’mUnprovable:Proposition<i>⇒I’mUnprovable:Proposition<i+1>, which is a contradiction.The crucial issue with the proofs in [Gödel 1931] is that the Gödel number of a proposition does not capture its order. Because of orders of propositions, the Diagonal Lemma [Gödel 1931] fails to construct the proposition I’mUnprovable.
See the following for more explanation: "Epistemology Cyberattacks" https://papers.ssrn.com/abstract=3603021
And a lot of the recent CPU performance gains are due to SIMD and multithreading, which are basically making CPUs more GPU-like, rather than the continuous improvement of serial performance as we've had up until ~2005.
I’d agree that it feels like it was intended for a more youthful audience, but I doubt I would have made all of the mental connections that I did without the life experience and broader knowledge of my 30s.
About the first third of the book was interesting in new ways of thinking about symbols and self-reference. After that it kept looping back around the same topics without really adding anything more. The dialogues were somewhat entertaining but I found myself wishing to cut past the rhetorical fluff and get to the point.
I've re-read it twice, once around the year 2000 an once this past year. It holds up well. This book really was a "major event" as the Pulitzer committee described it.
In the short run, expand then compress, expand then compress. In the long run, the size of the model will always compress toward the capacity of the most intelligent individual.
But an enormous part of the (useful but narrow) success of AI is a specific set of ML tech, especially supervised learning.
And the amount of computational horsepower and data required gives one at least some pause given what the human brain can do with apparently so much less. Which implies there are probably significant advances required in cognitive science and neurophysiology and perhaps other fields of studies we haven't even considered. We may need other ways to solve problems that have proven more elusive to brute force than it looked like 5 years or so ago (e.g. general purpose full self-driving in busy cities).
We (http://www.heurolabs.com) are contemplating open sourcing it, or at least publish the design documents
The dead horse that I like to beat is that there has been no progress in AI so far, the past 5-10 years of successes are all about mere perception. If you like the Kahneman "fast and slow" framework, almost without exception the systems we're seeing today are fast, "system 1" responders, which take input and need to make an instant decision. That's freaking awesome, and was way out of reach even 15 years ago, so I'm throwing no shade at all on that achievement! It's astounding how much turns out to be possible in systems like this - a priori, I never would have thought that transformers could do as much as they've proven to be able to, because frankly the architecture is super limited and it's super unlikely that humans are able to extract anywhere near as much meaning from an instant, non-reflective pass over a paragraph of text as GPT-3 does.
But there's a lot about those systems that makes them much easier to design and train, not the least of which is that descent via backprop works great as a training strategy when the input and target output are available at the same time. Real "system 2" thought can be spread over minutes, hour, or days, and I don't care how far you unroll backprop-through-time, you're not going to train that effectively without some other innovation by simply following the obvious error metrics. If we can get there we will almost certainly see big data lose its pedestal: it's great to have, but humans don't need to read the entire Internet to understand a web page, that's an artifact of forcing this to be done by a model that doesn't have dynamics.
I disagree with Hofstadter's view (at least when he wrote GEB and some of his other classics) that explicit abstract reasoning is the right way to solve any of this; my gut tells me that in the end we're going to end up still using some sort of neural architecture and abstract reasoning rules will be implicitly learned mostly locally based on internal "error" signals that guide state transitions. "Learning how to learn" is probably going to be a big part of that, because current systems don't learn, they are merely trained to perceive by brute forcing weights down the stack. But some serious shifts in research focus will be required in order to break through the weaknesses of today's systems, and they all point more in the direction of reasoning, which is woefully underprioritized in all but a very small handful of niche AI labs today.
At the least, you would learn which domains really do need new architectures and which are basically good enough when used as part of a larger system that can help compensate for the shortcomings.
Dawkins’ book The Selfish Gene, published just a few years earlier, is the clearest exposition that I have read of how this process probably played out in terrestrial life: the “semantics” encoded by amino acid sequences correspond to a molecule/cell/organism’s likelihood of surviving and replicating. For all but the simplest and most ephemeral replicators, this generally means accurately predicting environmental conditions. General intelligence, then, would conceivably arise simply due to selection pressures pushing organisms to live in the broadest range of environments.
In some sense, this process does sound more like an engineering problem, as the embodiment which “contains” the intelligence is probably not an optional component.
I'm still intrigued by the Neural Ordinary Differential Equations paper [0] and the research in that direction. I also remember reading about differentiable neural computers, but I haven't followed the progress of that at all.
To see an example of a task which requires actual artificial intelligence reasoning, see this article by none other than Hofstadter himself :p https://www.theatlantic.com/technology/archive/2018/01/the-s...
In 2021 the conventional wisdom is basically the opposite of the sentiment expressed here. Progress in AI is not coming from abstract reasoning. It is coming from an increase in raw power, driven by GPUs, and mathematical models that are designed more to harness large numbers and brute force searches for formulas, rather than a high-minded algorithmic embodiment of abstraction.
I loved GEB when I first read it in high school, and when I first reread it years later, but I don’t think its fundamental view of the relation between minds and machines has stood the test of time. It underestimated what behaviors could be emergent from running simple algorithms on large datasets. It is one of the most beautiful expressions of the ideas of the “classic” age of 1970’s AI, awesome to read but in the end somewhat incorrect about the future.
Perhaps one day the pendulum will swing back, and we will discover that large datasets are in some ways overrated, and clever aesthetic senses of pattern are necessary for progress. On that day it will be quite interesting to reread GEB.