News for hackers. Stuff that matters.
Re:Yann LeCun's comment on AlphaGo and true AI by pavanky on Monday March 14, 2016 @12:25AM (#11281059)
Can someone more knowledgeable explain why biological systems are considered unsupervised instead of reinforcement based systems?

While it seems intuitive that most individual "intelligent" systems in animals can be seen as unsupervised, isn't life itself driven in a reinforced manner?

Re:Yann LeCun's comment on AlphaGo and true AI by nabla9 on Monday March 14, 2016 @08:33AM (#11282438)
There is no enough reinforcement volume to learn anything complex.

If child gets external reward for every waking moment until she is 12 years old, it's just 4.2 million signals.

Reinforcement learning works for fine motor control and other tasks where the feedback loop is tight and immediate. Reinforcement and conditioning can also modulate high level cognition and behavior, but it's not the secret sauce of learning.

Re:Yann LeCun's comment on AlphaGo and true AI by sabertoothed on Monday March 14, 2016 @12:11PM (#11283652)
I could not follow your argument. Could you elaborate? (Honest question)
Re:Yann LeCun's comment on AlphaGo and true AI by nabla9 on Monday March 14, 2016 @01:48PM (#11284247)
Reinforcement learning is learning by interacting with an environment. RL agent learns from the consequences of its actions.

Biological system don't live long enough to get enough feedback to learn complex behavior trough consequences. Animal or human must be able to generalize and categorize what they have learned correctly without external feedback teaching it how to derive the function that's doing it.

For example, if you want to learn how to tie a complex knot and learn it trough trial and error you might have try it million times if you improve your behavior mainly trough consequences of your actions. In practice you probably try only 5-10 times before you learn to do it and it involves pausing and looking at the problem. There is some kind of unsupervised model building happening that is not involving external input.

Re:Yann LeCun's comment on AlphaGo and true AI by sabertoothed on Monday March 14, 2016 @02:34PM (#11284572)
Ok, I see what you mean now.

But it seems like (or one could misunderstand you in way that) you see those concepts as mutually exclusive. I would assume a combination of reinforcement learning and unsupervised (and supervised) learning.

Rats have been trained to detect landmines and then go back to their trainers and show them the mine. This is complex behaviour that was taught using reinforcement (at least on a top level). There will be some unsupervised learning going on in the rat's brain on a lower level. But it is complex behaviour and it's been reinforcement learnt.

Re:Yann LeCun's comment on AlphaGo and true AI by nabla9 on Monday March 14, 2016 @03:17PM (#11284860)
>see those concepts as mutually exclusive.

I certainly don't. Reinforcement or conditioning is part of it, but it's not the cake.

Re:Yann LeCun's comment on AlphaGo and true AI by rm999 on Monday March 14, 2016 @12:39AM (#11281111)
The brain needs to form structure before any sort of reinforcement learning is even possible. This structure comes from an "unsupervised" process we don't understand well.
Re:Yann LeCun's comment on AlphaGo and true AI by justicezyx on Monday March 14, 2016 @01:07AM (#11281175)
Isn't it from millions year of natural selection? The reason why a human brain can think, is just by chance.
Re:Yann LeCun's comment on AlphaGo and true AI by daveguy on Monday March 14, 2016 @01:57PM (#11284324)
Personally I think a lot of it has to do with evolution. We certainly have different sections of the brains that are more utilized in different tasks, auditory, visual, motor, etc. I think it is likely a lot of the initial feedback is preset -- the instinctual positive and negative responses of certain stimulus.
Re:Yann LeCun's comment on AlphaGo and true AI by conceit on Tuesday March 15, 2016 @04:36PM (#11292780)
As much as it doesn't help to call it intelligent design, chance is not any good either, if asking for a reason. Random chance is as good as no reason at all.
Re:Yann LeCun's comment on AlphaGo and true AI by justicezyx on Wednesday March 16, 2016 @01:52AM (#11295320)
Mixing natural selection and intelligent design is a problem, but that's not a problem for natural selection.
Re:Yann LeCun's comment on AlphaGo and true AI by conceit on Wednesday March 16, 2016 @09:37PM (#11302053)
What's the context? The GP probably meant that intelligence is an emergent property not directly coded in DNA.

Natural selection is not pure chance. At least seeing it that way doesn't yield anything except maybe lowered expectations. You want some expectations.

Hiding you theory behind Nature is not any less religious. Nature is pretty much synonym with living things, existence in general, and actually stems from to be born. Nature as the reason for things being born is thus kind of true by definition. The rational behind the generational evolution is the exact opposite of pure chance, though. Chance is only applicable if there are two possible outcomes. In hindsight, only one outcome was possible, so there are no chances, just facts.

The argument I am getting at is, self similarity seems to be such a common property, that it's maybe inevitable somewhere in a complex system. From there to self awareness and critical thinking is a long way to go, but what you call millions of years is a blink of an eye on a cosmological scale.

The idea of complexity includes enough variety to appear random to us, so we might agree that it's just a difficult to express concept and this is only semantic quibbling.

Re:Yann LeCun's comment on AlphaGo and true AI by baobabaobab on Monday March 14, 2016 @12:51AM (#11281137)
>While it seems intuitive that most individual "intelligent" systems in animals can be seen as unsupervised, isn't life itself driven in a reinforced manner?

I think apparently unsupervised systems could be explained by models that predict a future input given their current state and input. Correct predictions are reinforced. An RNN-like model.

(If the network is small, it should learn some compressed representation, which can be used as an input to a more abstract layer that makes more general predictions over a longer time period)

Re:Yann LeCun's comment on AlphaGo and true AI by daveguy on Monday March 14, 2016 @01:44PM (#11284221)
In AI the definition of reinforcement learning vs unsupervised is that reinforcement requires external identification of the correct result to compare with the given result.

If going through life you had artificial reality display that flashed "apple" in front of every apple and "orange" in front of every orange you would have reinforcement learning. There is certainly a major component of reinforcement in teaching, but that does not account for all of learning.

With humans we are able to transfer concepts to different situations without examples and there are a lot of categories that are identified without any explicitly reinforced examples.

Re:Yann LeCun's comment on AlphaGo and true AI by zardo on Monday March 14, 2016 @03:49PM (#11285064)
You're describing supervised learning. Reinforcement learning is more like you stick random things in your mouth, and if one tastes good, you learn to stick that kind of thing in your mouth more often.
Re:Yann LeCun's comment on AlphaGo and true AI by daveguy on Monday March 14, 2016 @04:20PM (#11285286)
Well, yes, I was describing supervised learning (as what doesn't happen) where labels are not present is unsupervised (which makes things much more difficult). Reinforcement is more like a special case between supervised and unsupervised. Difficult like unsupervised because the labels aren't known, but with some feedback that is not a direct answer. Sorry for the confusion. There is very little in the way of purely unsupervised learning. I think the distinction between unsupervised and reinforcement is not very well defined. I would argue that even a "pure" unsupervised algorithm like k-means has a kind of reinforcement in the group means.
Re:Yann LeCun's comment on AlphaGo and true AI by sabertoothed on Monday March 14, 2016 @12:15PM (#11283669)
I would agree.

Could one not argue that even unsupervised learning is kind of reinforced by let's say the emotion you obtain from successfully performing a learning task? Then the reward signal does not come from the external environment but from resolving cognitive dissonance in the brain.

Happy to be proved wrong by experts here.