News for hackers. Stuff that matters.
Re:Why traditional reinforcement learning will probably not ... by bra-ket on Wednesday October 21, 2020 @03:37PM (#24851239)
I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent.

Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient model of learning as reward and punishment, iterative trial and error process.

This 'Skinnerism' has been discredited in cognitive psychology decades ago and makes absolutely no biological sense whatsoever for the simple reason that any organism trying to adapt in this way will be eaten by predators before minimizing its "error function" sufficiently.

Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuristics and hardcoded biases instead of doing some kind of brute force optimization.

This is the case where computational efficiency [1] comes first and sets the constraints by which cognitive apparatus needs to be developed.

As for actual cognition models a good place to start is not ML/AI field (which tends to getting stuck in local minima as a whole), but state-of-the-art cognitive psychology, and may be looking at research in "distributional semantics", "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience.

You'd be surprised how much knowledge we gained about the mind since those RL & ANN models developed in the 1940s.

[1] https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...

Re:Why traditional reinforcement learning will probably not ... by postnihilism on Wednesday October 21, 2020 @05:13PM (#24852245)
> "This 'Skinnerism' has been discredited in cognitive psychology decades ago and makes absolutely no biological sense whatsoever for the simple reason that any organism trying to adapt in this way will be eaten by predators before minimizing its "error function" sufficiently."

> "Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuristics and hardcoded biases instead of doing some kind of brute force optimization."

But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured.

Re:Why traditional reinforcement learning will probably not ... by qyv on Wednesday October 21, 2020 @05:44PM (#24852552)
> But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured.

This is true in the context of the universe as a whole, not by the organism itself.

Re:Why traditional reinforcement learning will probably not ... by renjimen on Wednesday October 21, 2020 @06:07PM (#24852763)
Except no organism is born a blank slate. Parent is correct in that our prior was massively expensive to construct
Re:Why traditional reinforcement learning will probably not ... by dkersten on Wednesday October 21, 2020 @09:21PM (#24854040)
So we can expect our ANN’s to yield AGI in a few million or billion years? That doesn’t sound like a good place to put our current efforts then.
Re:Why traditional reinforcement learning will probably not ... by mannykannot on Wednesday October 21, 2020 @09:41PM (#24854129)
That does not necessarily follow, as I imagine you well know.
Re:Why traditional reinforcement learning will probably not ... by YeGoblynQueenne on Thursday October 22, 2020 @06:56AM (#24856713)
Why wouldn't it follow? Human intelligence evolved in the real world with all its vast information content. Deep learning systems are only trained on a few terrabytes of data of a single type (images, text, sound etc). Even if they can be trained faster than the rate at which animals evolved, their training data is so poor, compared to the "data" that "trained" animal intelligence that we'll be lucky if we can arrive at anything comparable to animal intelligence by deep learning in a billion years.

Or unlucky, as the case may be.

Re:Why traditional reinforcement learning will probably not ... by mannykannot on Thursday October 22, 2020 @08:47AM (#24857419)
You elided the "necessarily".

One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.

Re:Why traditional reinforcement learning will probably not ... by dkersten on Thursday October 22, 2020 @10:22AM (#24858317)
Not grounds for stopping work[1], but perhaps grounds to explore other avenues[2] to see if something else might yield faster results.

I’m no expert, but my personal opinion is that AGI will probably be some hybrid approach that uses some reinforcement learning mixed with other techniques. At the very least, I think an AGI will need to exist in an interactive environment rather than just trained on preset datasets. Prior context or not, a child doesn’t learn by being shown a lot of images, it learns by being able to poke at the world to see what happens. I think an AGI will likely require some aspect of that (and apply reinforcement learning that way).

But like I said, I’m no expert and that’s just my layperson opinion.

[1] if the goal is AGI, if it’s not then of course there’s no reason to stop

[2] some people are doing just that, of course

Re:Why traditional reinforcement learning will probably not ... by mannykannot on Thursday October 22, 2020 @09:20PM (#24865157)
Fair enough, though I do not think the evidence from evolution moves the needle much with respect to the timeline. For one thing, evolution was not dedicated to the achievement of intelligence.
Re:Why traditional reinforcement learning will probably not ... by dkersten on Friday October 23, 2020 @05:22PM (#24874332)
Sounds reasonable.
Re:Why traditional reinforcement learning will probably not ... by YeGoblynQueenne on Thursday October 22, 2020 @10:48AM (#24858580)
>> You elided the "necessarily".

Well, if it follows, then it follows necessarily. But maybe that's just a deformation professionelle? I spend a lot of time working with automated theorem proving where there's no ifs and buts about conclusions following from premises.

Re:Why traditional reinforcement learning will probably not ... by mannykannot on Thursday October 22, 2020 @09:16PM (#24865145)
If I am not mistaken, it does not necessarily follow unless it turns out to be a sound argument in every possible world.
Re:Why traditional reinforcement learning will probably not ... by YeGoblynQueenne on Friday October 23, 2020 @05:06AM (#24867380)
Ah, so you are making a formal argument? In that case you should stick to formal language. And probably publish it in a different venue :)
Re:Why traditional reinforcement learning will probably not ... by mannykannot on Friday October 23, 2020 @08:32AM (#24868753)
No, I am simply responding to your rather formal point, in kind. Unless you are aguing for it being an established fact that the time evolution took to produce intelligent life rules out any form of reinforcement learning producing AI in any remotely reasonable period of time, then that original point of yours does not seem to be going anywhere.

In your work on theorem proving, am I right in guessing that there are no 'ifs' or 'buts' because the truth of premises is not an issue? In the "evolution argument", the premises/lemmas are not just that evolution took a long time, but also something along the lines of significant speedup not being possible.

You might notice that in another comment, I suggested that we might still be in the AI Cambrian. I'm not being inconsistent, as no-one knows for sure one way or the other.

Re:Why traditional reinforcement learning will probably not ... by YeGoblynQueenne on Friday October 23, 2020 @08:54AM (#24868931)
I didn't make a formal point- my comment is a comment on an internet message board, where it's very unlikely to find formal arguments being made. But perhaps we do not agree on what constitutes a "(rather) formal point"? I made a point in informal language and in a casual manner and as part of an informal discussion ... on Hacker News. We are not going to prove or disprove any theorems here.

But, to be sure, as is common when this kind of informal conversation suddendly sprouts semi-formal language, like "argument", "claim", "proof", "necessarily follows" etc, I am not even sure what exactly it is we are arguing about, anymore. What exactly is your disagreement with my comment? Could you please explain?

Re:Why traditional reinforcement learning will probably not ... by mannykannot on Friday October 23, 2020 @10:05AM (#24869611)
"Necessarily" has general usage as well, you know... why would you read it otherwise, especially given the reasonable observation you make about this site? And my original point is not actually wrong, either: whether reinforcement learning will proceed at the pace of evolution is a topic of speculation - it is possible that it will, and possible that it will not.

Insofar is I have an issue with your comment, it is that it is not going anywhere, as I explained in my previous post.

Re:Why traditional reinforcement learning will probably not ... by YeGoblynQueenne on Friday October 23, 2020 @07:24PM (#24875464)
>> Insofar is I have an issue with your comment, it is that it is not going anywhere, as I explained in my previous post.

I see this god-moding of my comment as a pretend-polite way to tell me I'm takling nonsense, that seems to be designed to avoid criticism for being rude to one's interlocutor on a site that has strong norms against that sort of thing, but without really trying to understand why those norms exist, i.e. because they make for more productive conversations and less wasting of everyone's time.

You made a comment to say that unless I claim that X (which you came up with), then my comment is not going anywhere. The intellectually corteous and honest response to a comment with which one does not agree is to try and understand the reasoning of the comment. Not to claim that there is only one possible explanation and therefore the comment must be wrong. That is just a straw man in sheep's clothing.

And this is not surprising given that it comes at the heels of nitpicking about supposedly important terminology (necessarily!). This is how discussions like this one go, very often. And that's why they should be avoided, because they just waste everyone's time.

Re:Why traditional reinforcement learning will probably not ... by mannykannot on Saturday October 24, 2020 @10:14AM (#24879328)
"Necessarily", when read according to your own expectations for this forum, made an important difference to my original post (without it, I would have been insisting that the issue is settled already), so it was reasonable for me to point out its removal. The nitpicking over it began with your response to me doing so, and you have kept it going by taking the worst possible reading of what I write. This is, indeed, how things sometimes go.

Meanwhile, in a branching thread, I had a short discussion with the author of the post I originally replied to, in which I agreed with the points he made there. Both of us, I think, clarified our positions and reached common ground. That is how it is supposed to go.

I did not set out to pick a fight with you, and if I had anticipated how you would take my words, I would have phrased things more clearly.

Re:Why traditional reinforcement learning will probably not ... by AlphaSite on Wednesday October 21, 2020 @07:29PM (#24853436)
I think the point huh is being mad is that individual people (or models) dot learn that way. It’s not like models training models, all the way down.
Re:Why traditional reinforcement learning will probably not ... by the8472 on Wednesday October 21, 2020 @09:12PM (#24853997)
Individual people are not trained from scratch. ML models often have to be (modulo fine-tuning) since the field is still young.
Re:Why traditional reinforcement learning will probably not ... by mannykannot on Wednesday October 21, 2020 @09:45PM (#24854142)
That's already changing. That we have only relatively recently moved beyond always starting from scratch might indicate that we are still in the Cambrian of AI, however...
Re:Why traditional reinforcement learning will probably not ... by GregarianChild on Wednesday October 21, 2020 @05:19PM (#24852311)
Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL.

   cognitive psychology
You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience?

[1] https://arxiv.org/abs/1505.00521

Re:Why traditional reinforcement learning will probably not ... by YeGoblynQueenne on Wednesday October 21, 2020 @06:14PM (#24852825)
Note that Turing complete means undecidable. An algorithm that can learn any program that can be computed by a Universal Turing Machine must, in the worst case, search an infinite program space. So, even if a Neural Turing Machine can learn arbitrary programs (I haven't read the paper so I can't say) it might need to consume infinite resources before learning any particular progarm.

In short- Turing completeness is no guaranteed path to AGI. Assuming an "AGI program" exists, it is hidden away in an infinity of almost identical, but not quite, programs.

Re:Why traditional reinforcement learning will probably not ... by bra-ket on Wednesday October 21, 2020 @08:54PM (#24853922)
>Where is .. AI

it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents.

I mean you can't just throw things at the wall and hope they stick, but it's literally the state of the art, if you follow ML conferences and their world-beating toy benchmarks results, with a lot of pseudo-rigorous handwaving for theory.

The reason physics has been so successful is that their theory closely followed empirical data and constraints imposed by nature.

I think the only hope to achieve common sense in AI is to align it with hard constraints living organisms have, using those constraints as a guide.

A few terms I mentioned are coming from that POV, if you dig a bit deeper they all have direct physical manifestation in natural learning systems.

Re:Why traditional reinforcement learning will probably not ... by YeGoblynQueenne on Thursday October 22, 2020 @06:49AM (#24856659)
>> it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents.

A small correction: that's deep learning researches, not AI researchers and not all machine learning researchers even. To be charitable, it's not even all deep learning researchers. It's just that the field of deep learning research has been inundated with new entrants who are sufficiently skilled to grok the practicalities but lack understanding of AI scholarship and produce unfortunately shoddy work that does not advance the field (any field, any of the aforementioned ones).

As a personal example, my current PhD studies are in Inductive Logic Programming which is, in short, machine-learning of logic programs (you know, Prolog etc). I would not be able to publish any papers without a theoretical section with actual theoretical results (i.e. theorems and their proofs - and it better be a theorem other than "more parameters beget better accuracy", which is not really a theorem). Reviewers would just reject such a paper without second thought, regardless of how many leaderboards I beat in my empirical results section.

And of course there are all the other fields of AI were work continues - search, classical planning, constraint satisfaction, automated theorem proving, knowledge engineering and so on and so forth.

Bottom line- the shoddy scholarship you flag up does not characterise the field of AI research, as a whole, it only afflicts a majority of modern deep learning research.

Re:Why traditional reinforcement learning will probably not ... by eli_gottlieb on Wednesday October 21, 2020 @10:41PM (#24854402)
>Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL.

Brainfuck is also Turing complete, so logically if we just do, for instance, Markov chain Monte Carlo for Bayesian program learning in Brainfuck, we can realize AGI that way.

"Everything is possible, but nothing is easy." The Turing tarpit.

Re:Why traditional reinforcement learning will probably not ... by ForHackernews on Wednesday October 21, 2020 @05:40PM (#24852514)
> Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL.

This seems like overstating your point. Nobody has been able to rigorously define "AI" yet, so there's no way of saying whether it's possible with a Turing machine architecture. The human brain, at least, doesn't seem that similar to a Turing architecture. Neurons don't carry out anything like discrete operations.

Maybe it's possible to run AGI on a Turing machine, maybe it's not, but there are more options than simply "possible with a Turing machine" or "completely impossible".

Re:Why traditional reinforcement learning will probably not ... by GregarianChild on Wednesday October 21, 2020 @05:55PM (#24852666)
While I agree that we don't have an agreed upon definition of AI, the problem is firmly in the "I" part of AI! The "A" part is taken to mean implementable by a computer, i.e. a Turing machine. This is the content of the Church–Turing thesis [1].

[1] https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis

Re:Why traditional reinforcement learning will probably not ... by ForHackernews on Thursday October 22, 2020 @04:42AM (#24855999)
> It states that a function on the natural numbers can be calculated by an effective method if and only if it is computable by a Turing machine.

The article at the top of this thread is specifically about properties ("The Generalized Archimedean Property") that real numbers do not possess.

There's also a little bit of slipperiness around the use of "AI" vs. "AGI" - you could easily argue (and people do!) that we've already achieved "AI" for many specialized domains. It's the General bit that seems to be the sticking point, and that this article focuses on.

Re:Why traditional reinforcement learning will probably not ... by joe_the_user on Wednesday October 21, 2020 @06:03PM (#24852735)
Maybe it's possible to run AGI on a Turing machine, maybe it's not,

Arguing from ignorance, of course nothing is knowable for certain. However there has been a lot of work on the universality of Turing machines, showing that a Turing machine can simulate any conceivable concept of finite computation and can approximate any conventional physical system.

I think a more useful way to express your intuition is to note that if human-built AGI comes into existence, it might be runnable on a Turing machine but quite possibly not efficiently so.

Re:Why traditional reinforcement learning will probably not ... by gnulinux on Wednesday October 21, 2020 @05:18PM (#24852301)
> The idea of using "rewards" as a learning mechanism and a path to actual cognition is just wrong, full stop.

I'm a layman (just a software engineer) but am curious, I train my cat only with rewards (never punishment because apparently doesn't work on cats) and the kitty learned how to high-five me, sit, jump, follow me etc. It seems to work really well for us. Basically, ever time he does something desirable, I click my pen and give him his favorite treats. Is this ineffective?

Re:Why traditional reinforcement learning will probably not ... by benchaney on Wednesday October 21, 2020 @05:30PM (#24852426)
Your cat was already capable cognition before you started training it. GP is talking about generating a cognition where it did not previously exist.
Re:Why traditional reinforcement learning will probably not ... by joe_the_user on Wednesday October 21, 2020 @05:51PM (#24852638)
Why couldn't a mechanism (say A Neural Turing or whatever) that you train be "cognition capable" when you start and then be trained to actual behavior after that?
Re:Why traditional reinforcement learning will probably not ... by benchaney on Wednesday October 21, 2020 @07:54PM (#24853614)
You would need something that is "cognition capable" first and that has not been invented yet.
Re:Why traditional reinforcement learning will probably not ... by joe_the_user on Thursday October 22, 2020 @12:47AM (#24854981)
It's hard to know. Maybe something "cognition cable" exists, it's just the proper train routine hasn't been provided to it.

But regardless, the broader point is yeah, combine something akin cognition capability and the proper training routine and there you go, AGI from "reinforcement learning", broadly defined.

Re:Why traditional reinforcement learning will probably not ... by kypro on Wednesday October 21, 2020 @05:52PM (#24852643)
Also a layman, but I think OPs point wasn't that it isn't possible, but that's it's not effective or analogous to how humans or other species learn.

For example, your cat's brain isn't just a randomly initialised neural net. Your cat comes pre-wired in such a way that it understands certain things about its environment and has certain innate biases that allow you to train it to do simple tricks with relative ease through a reward mechanism.

A more analogous example would be building a cat-like robot with four legs and a neural processor then switching it on and expecting to be able to train it with treats. Without a useful initial neural state (founded with an understanding of cognitive psychology and neuroscience) it would be almost totally useless.

Re:Why traditional reinforcement learning will probably not ... by anotherProd on Wednesday October 21, 2020 @05:00PM (#24852127)
As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm.

Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" is a pretty accurate description with how far we've come when it comes to consciousness, and I would imagine even less for the newer field of AI/AGI (consciousness has been around for a while P: and our theories have mostly sidestepped this real world phenomenon).

> "The idea of using "rewards" as a learning mechanism and a path to actual cognition is just wrong, full stop."

This to me is a huge red flag (mostly of ego/hubris). I think if we rephrased the goal to not talk about "AGI" and maybe around quantitative things like the things you've listed ("computational efficiency", likelihood of being stuck in local minimimas, etc) then I'd happily concede that we should be looking at "X" and not "Y" but unless I've missed something, again likely, when we're talking about AGI, we're talking about consciousness (epiphenomenon that come about through physical/deterministic interactions). A quick way to gut check myself here is twisting what you state is not a good place to start "ML/AI field... gets stuck in local minima" and ask myself is it possible that local minima (which we consider "bad" for current/traditional tasks) could be necessary for consciousness ? I think the widely accepted answer to this is currently "We don't know".

If I think that achieving AGI is going to be similar to what the algorithms and architecture we currently use (where the likelihood of being stuck in a local minima is something we can look at) then sure, your opinions stand. But that is just a guess and unless I'm mistaken AGI hasn't been achieved because we don't know how to do it.

This isn't to say that we should have 100% of the data before making strong judgements like this about a subject. It's just that the subject of "consciousness" is a big one (I'd say THE big one) so making such strong statements about something we know we don't know much about is interesting. <- this is where I get flashbacks to SE world where a missing piece of data can really throw you off or leads to wrong assumptions and when I think about consciousness we know we don't know a lot.

Re:Why traditional reinforcement learning will probably not ... by thomasfromcdnjs on Wednesday October 21, 2020 @09:13PM (#24854006)
Stumbled upon this the other day, seems interesting ¯\_(ツ)_/¯

https://en.wikipedia.org/wiki/The_Emperor%27s_New_Mind

"Penrose argues that human consciousness is non-algorithmic, and thus is not capable of being modeled by a conventional Turing machine, which includes a digital computer. Penrose hypothesizes that quantum mechanics plays an essential role in the understanding of human consciousness. The collapse of the quantum wavefunction is seen as playing an important role in brain function."

Re:Why traditional reinforcement learning will probably not ... by thechao on Thursday October 22, 2020 @08:46AM (#24857400)
Penrose is a superlatively brilliant physicist, but his opinion on AI is the worst sort of woo. It’s little better than reading Chopra. His argument is most salient as an example of attempting to use authority in one field to garner authority in another.
Re:Why traditional reinforcement learning will probably not ... by bra-ket on Wednesday October 21, 2020 @05:30PM (#24852428)
My point was that RL/DL is being used like some kind of massive hammer to hit all the nails. Cognition requires different, specialized, energy-efficient tools.

> consciousness

All talk about this is premature and "pre-science", before we figure out more basic, fundamental things like object storage and recall from memory, object recognition from sensory input, concept representation and formation, the exact mechanism of "chunking" [1], "translational invariance" [2], generalization along concept hierarchy and different scales, representation of causal structures, proximity search and heuristics, innate coordinate system, innate "grammar".

Even having a working, biologically-plausible model of navigation in 3d spaces by mice, without spending a ton of energy training the model, would be a good first step. In fact there is evidence that navigational capacity [3] is the basis of more abstract forms of thinking.

On all of these things we have decades worth of research and widely published, fundamental, Nobel-winning discoveries which are almost completely ignored by the AI field stuck in its comfort zone. Saying "we have no idea" is just being lazy.

Edit: As for OP's actual paper I think something like complex-valued RL [4] might bypass his main claims entirely. But my point is that RL itself is a dead end, trivializing the problem at hand.

[1] https://en.wikipedia.org/wiki/Chunking_(psychology)

[2] http://www.moreisdifferent.com/2017/09/hinton-whats-wrong-wi...

[3] http://www.scholarpedia.org/article/Grid_cells

[4] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22c...

Re:Why traditional reinforcement learning will probably not ... by nagaa on Thursday October 22, 2020 @08:22AM (#24857218)
I have learned to play piano and drive a car. IMO both of these took two completely different sets of systems and algorithms in order to accomplish the learning task. Nothing I learned from piano applies to driving and vice versa. The only thing in common is my brain. We want a computer though to apply those algorithms I learned driving and playing piano to golf and have it work. We will then have "AGI". Obviously that fails. Obviously.
Re:Why traditional reinforcement learning will probably not ... by xvilka on Wednesday October 21, 2020 @09:00PM (#24853943)
I think it will require at least a few more centuries to build AGI.
Re:Why traditional reinforcement learning will probably not ... by eli_gottlieb on Wednesday October 21, 2020 @10:39PM (#24854397)
>state-of-the-art cognitive psychology, and may be looking at research in "distributional semantics", "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience.

Look, uh, I've read Gardenfors too, but are those really the state of the art? I don't remember there being anything about them at CogSci this past summer. Maybe I wasn't paying close-enough attention?

Re:Why traditional reinforcement learning will probably not ... by bra-ket on Thursday October 22, 2020 @01:30AM (#24855174)
In RL/DL context any CogSci developments after 1943 is the state of the art.

Some interesting recent work [1] related to Gardenfors ideas was combining them with discovery of place & grid cells, and extending the "cognitive maps" and spatial navigation machinery into concept spaces, treating the innate coordinate system as foundation for abstraction and generalization facilities.

And they actually found empirical data to prove it in [1] and related papers, so Gardenfors was right.

I believe it gotta be the starting point for anyone seriously considering an AI, kind of like Cartesian foundation. It also aligns nicely with rich "distributional semantics" work and popular vector space models.

[1] https://pubmed.ncbi.nlm.nih.gov/27313047/

Re:Why traditional reinforcement learning will probably not ... by odnes on Wednesday October 21, 2020 @05:52PM (#24852650)
> The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent.

A lot of research in RL is focused on intrinsic motivation and the question of whether we can bootstrap our own 'rewards' from our ability to predict and control the future according to some self-defined goals/hypotheses.

Re:Why traditional reinforcement learning will probably not ... by cmrx64 on Wednesday October 21, 2020 @06:00PM (#24852703)
do you know about the dopamine reward error hypothesis? https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6721851/ is it so wrong? what does cognitive psychology have to say about how these neurons work? this is a lot more recent than the 40s and behaviorism.
Re:Why traditional reinforcement learning will probably not ... by bra-ket on Wednesday October 21, 2020 @07:11PM (#24853295)
dopamine rewards operate on a different time scale vs. that required by these error correction models. I don't remember the exact paper, will need to look it up, but it was orders of magnitude difference in response times.

Edit: for authoritative reference on biologically-plausible learning see anything by Edmund Rolls [1]. He explicitly stated in his recent book [2] that something like back-propagation, or similar error correction mechanisms have no supporting evidence in experimental data collected so far

[1] https://www.oxcns.org/profile.html

[2] https://www.amazon.com/Cerebral-Cortex-Principles-Edmund-Rol...

Re:Why traditional reinforcement learning will probably not ... by cmrx64 on Tuesday October 27, 2020 @03:03PM (#24910647)
thanks for the link, it's been a lovely rabbit hole :)