> "Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuristics and hardcoded biases instead of doing some kind of brute force optimization."
But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured.
This is true in the context of the universe as a whole, not by the organism itself.
Or unlucky, as the case may be.
One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.
I’m no expert, but my personal opinion is that AGI will probably be some hybrid approach that uses some reinforcement learning mixed with other techniques. At the very least, I think an AGI will need to exist in an interactive environment rather than just trained on preset datasets. Prior context or not, a child doesn’t learn by being shown a lot of images, it learns by being able to poke at the world to see what happens. I think an AGI will likely require some aspect of that (and apply reinforcement learning that way).
But like I said, I’m no expert and that’s just my layperson opinion.
[1] if the goal is AGI, if it’s not then of course there’s no reason to stop
[2] some people are doing just that, of course
Well, if it follows, then it follows necessarily. But maybe that's just a deformation professionelle? I spend a lot of time working with automated theorem proving where there's no ifs and buts about conclusions following from premises.
In your work on theorem proving, am I right in guessing that there are no 'ifs' or 'buts' because the truth of premises is not an issue? In the "evolution argument", the premises/lemmas are not just that evolution took a long time, but also something along the lines of significant speedup not being possible.
You might notice that in another comment, I suggested that we might still be in the AI Cambrian. I'm not being inconsistent, as no-one knows for sure one way or the other.
But, to be sure, as is common when this kind of informal conversation suddendly sprouts semi-formal language, like "argument", "claim", "proof", "necessarily follows" etc, I am not even sure what exactly it is we are arguing about, anymore. What exactly is your disagreement with my comment? Could you please explain?
Insofar is I have an issue with your comment, it is that it is not going anywhere, as I explained in my previous post.
I see this god-moding of my comment as a pretend-polite way to tell me I'm takling nonsense, that seems to be designed to avoid criticism for being rude to one's interlocutor on a site that has strong norms against that sort of thing, but without really trying to understand why those norms exist, i.e. because they make for more productive conversations and less wasting of everyone's time.
You made a comment to say that unless I claim that X (which you came up with), then my comment is not going anywhere. The intellectually corteous and honest response to a comment with which one does not agree is to try and understand the reasoning of the comment. Not to claim that there is only one possible explanation and therefore the comment must be wrong. That is just a straw man in sheep's clothing.
And this is not surprising given that it comes at the heels of nitpicking about supposedly important terminology (necessarily!). This is how discussions like this one go, very often. And that's why they should be avoided, because they just waste everyone's time.
Meanwhile, in a branching thread, I had a short discussion with the author of the post I originally replied to, in which I agreed with the points he made there. Both of us, I think, clarified our positions and reached common ground. That is how it is supposed to go.
I did not set out to pick a fight with you, and if I had anticipated how you would take my words, I would have phrased things more clearly.
cognitive psychology
You are overselling the insights of this discipline.
Has cognitive psychology solved its replication problems?
Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience?In short- Turing completeness is no guaranteed path to AGI. Assuming an "AGI program" exists, it is hidden away in an infinity of almost identical, but not quite, programs.
it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents.
I mean you can't just throw things at the wall and hope they stick, but it's literally the state of the art, if you follow ML conferences and their world-beating toy benchmarks results, with a lot of pseudo-rigorous handwaving for theory.
The reason physics has been so successful is that their theory closely followed empirical data and constraints imposed by nature.
I think the only hope to achieve common sense in AI is to align it with hard constraints living organisms have, using those constraints as a guide.
A few terms I mentioned are coming from that POV, if you dig a bit deeper they all have direct physical manifestation in natural learning systems.
A small correction: that's deep learning researches, not AI researchers and not all machine learning researchers even. To be charitable, it's not even all deep learning researchers. It's just that the field of deep learning research has been inundated with new entrants who are sufficiently skilled to grok the practicalities but lack understanding of AI scholarship and produce unfortunately shoddy work that does not advance the field (any field, any of the aforementioned ones).
As a personal example, my current PhD studies are in Inductive Logic Programming which is, in short, machine-learning of logic programs (you know, Prolog etc). I would not be able to publish any papers without a theoretical section with actual theoretical results (i.e. theorems and their proofs - and it better be a theorem other than "more parameters beget better accuracy", which is not really a theorem). Reviewers would just reject such a paper without second thought, regardless of how many leaderboards I beat in my empirical results section.
And of course there are all the other fields of AI were work continues - search, classical planning, constraint satisfaction, automated theorem proving, knowledge engineering and so on and so forth.
Bottom line- the shoddy scholarship you flag up does not characterise the field of AI research, as a whole, it only afflicts a majority of modern deep learning research.
Brainfuck is also Turing complete, so logically if we just do, for instance, Markov chain Monte Carlo for Bayesian program learning in Brainfuck, we can realize AGI that way.
"Everything is possible, but nothing is easy." The Turing tarpit.
This seems like overstating your point. Nobody has been able to rigorously define "AI" yet, so there's no way of saying whether it's possible with a Turing machine architecture. The human brain, at least, doesn't seem that similar to a Turing architecture. Neurons don't carry out anything like discrete operations.
Maybe it's possible to run AGI on a Turing machine, maybe it's not, but there are more options than simply "possible with a Turing machine" or "completely impossible".
[1] https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis
The article at the top of this thread is specifically about properties ("The Generalized Archimedean Property") that real numbers do not possess.
There's also a little bit of slipperiness around the use of "AI" vs. "AGI" - you could easily argue (and people do!) that we've already achieved "AI" for many specialized domains. It's the General bit that seems to be the sticking point, and that this article focuses on.
Arguing from ignorance, of course nothing is knowable for certain. However there has been a lot of work on the universality of Turing machines, showing that a Turing machine can simulate any conceivable concept of finite computation and can approximate any conventional physical system.
I think a more useful way to express your intuition is to note that if human-built AGI comes into existence, it might be runnable on a Turing machine but quite possibly not efficiently so.
I'm a layman (just a software engineer) but am curious, I train my cat only with rewards (never punishment because apparently doesn't work on cats) and the kitty learned how to high-five me, sit, jump, follow me etc. It seems to work really well for us. Basically, ever time he does something desirable, I click my pen and give him his favorite treats. Is this ineffective?
But regardless, the broader point is yeah, combine something akin cognition capability and the proper training routine and there you go, AGI from "reinforcement learning", broadly defined.
For example, your cat's brain isn't just a randomly initialised neural net. Your cat comes pre-wired in such a way that it understands certain things about its environment and has certain innate biases that allow you to train it to do simple tricks with relative ease through a reward mechanism.
A more analogous example would be building a cat-like robot with four legs and a neural processor then switching it on and expecting to be able to train it with treats. Without a useful initial neural state (founded with an understanding of cognitive psychology and neuroscience) it would be almost totally useless.
Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" is a pretty accurate description with how far we've come when it comes to consciousness, and I would imagine even less for the newer field of AI/AGI (consciousness has been around for a while P: and our theories have mostly sidestepped this real world phenomenon).
> "The idea of using "rewards" as a learning mechanism and a path to actual cognition is just wrong, full stop."
This to me is a huge red flag (mostly of ego/hubris). I think if we rephrased the goal to not talk about "AGI" and maybe around quantitative things like the things you've listed ("computational efficiency", likelihood of being stuck in local minimimas, etc) then I'd happily concede that we should be looking at "X" and not "Y" but unless I've missed something, again likely, when we're talking about AGI, we're talking about consciousness (epiphenomenon that come about through physical/deterministic interactions). A quick way to gut check myself here is twisting what you state is not a good place to start "ML/AI field... gets stuck in local minima" and ask myself is it possible that local minima (which we consider "bad" for current/traditional tasks) could be necessary for consciousness ? I think the widely accepted answer to this is currently "We don't know".
If I think that achieving AGI is going to be similar to what the algorithms and architecture we currently use (where the likelihood of being stuck in a local minima is something we can look at) then sure, your opinions stand. But that is just a guess and unless I'm mistaken AGI hasn't been achieved because we don't know how to do it.
This isn't to say that we should have 100% of the data before making strong judgements like this about a subject. It's just that the subject of "consciousness" is a big one (I'd say THE big one) so making such strong statements about something we know we don't know much about is interesting. <- this is where I get flashbacks to SE world where a missing piece of data can really throw you off or leads to wrong assumptions and when I think about consciousness we know we don't know a lot.
https://en.wikipedia.org/wiki/The_Emperor%27s_New_Mind
"Penrose argues that human consciousness is non-algorithmic, and thus is not capable of being modeled by a conventional Turing machine, which includes a digital computer. Penrose hypothesizes that quantum mechanics plays an essential role in the understanding of human consciousness. The collapse of the quantum wavefunction is seen as playing an important role in brain function."
> consciousness
All talk about this is premature and "pre-science", before we figure out more basic, fundamental things like object storage and recall from memory, object recognition from sensory input, concept representation and formation, the exact mechanism of "chunking" [1], "translational invariance" [2], generalization along concept hierarchy and different scales, representation of causal structures, proximity search and heuristics, innate coordinate system, innate "grammar".
Even having a working, biologically-plausible model of navigation in 3d spaces by mice, without spending a ton of energy training the model, would be a good first step. In fact there is evidence that navigational capacity [3] is the basis of more abstract forms of thinking.
On all of these things we have decades worth of research and widely published, fundamental, Nobel-winning discoveries which are almost completely ignored by the AI field stuck in its comfort zone. Saying "we have no idea" is just being lazy.
Edit: As for OP's actual paper I think something like complex-valued RL [4] might bypass his main claims entirely. But my point is that RL itself is a dead end, trivializing the problem at hand.
[1] https://en.wikipedia.org/wiki/Chunking_(psychology)
[2] http://www.moreisdifferent.com/2017/09/hinton-whats-wrong-wi...
[3] http://www.scholarpedia.org/article/Grid_cells
[4] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22c...
Look, uh, I've read Gardenfors too, but are those really the state of the art? I don't remember there being anything about them at CogSci this past summer. Maybe I wasn't paying close-enough attention?
Some interesting recent work [1] related to Gardenfors ideas was combining them with discovery of place & grid cells, and extending the "cognitive maps" and spatial navigation machinery into concept spaces, treating the innate coordinate system as foundation for abstraction and generalization facilities.
And they actually found empirical data to prove it in [1] and related papers, so Gardenfors was right.
I believe it gotta be the starting point for anyone seriously considering an AI, kind of like Cartesian foundation. It also aligns nicely with rich "distributional semantics" work and popular vector space models.
A lot of research in RL is focused on intrinsic motivation and the question of whether we can bootstrap our own 'rewards' from our ability to predict and control the future according to some self-defined goals/hypotheses.
Edit: for authoritative reference on biologically-plausible learning see anything by Edmund Rolls [1]. He explicitly stated in his recent book [2] that something like back-propagation, or similar error correction mechanisms have no supporting evidence in experimental data collected so far
[1] https://www.oxcns.org/profile.html
[2] https://www.amazon.com/Cerebral-Cortex-Principles-Edmund-Rol...
Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient model of learning as reward and punishment, iterative trial and error process.
This 'Skinnerism' has been discredited in cognitive psychology decades ago and makes absolutely no biological sense whatsoever for the simple reason that any organism trying to adapt in this way will be eaten by predators before minimizing its "error function" sufficiently.
Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuristics and hardcoded biases instead of doing some kind of brute force optimization.
This is the case where computational efficiency [1] comes first and sets the constraints by which cognitive apparatus needs to be developed.
As for actual cognition models a good place to start is not ML/AI field (which tends to getting stuck in local minima as a whole), but state-of-the-art cognitive psychology, and may be looking at research in "distributional semantics", "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience.
You'd be surprised how much knowledge we gained about the mind since those RL & ANN models developed in the 1940s.
[1] https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...