News for hackers. Stuff that matters.
Re:AlphaGo Zero: Learning from scratch by Houshalter on Wednesday October 18, 2017 @05:47PM (#15503511)
As far as we know the brain is just a "linear algebra blackbox". It's an uninteresting reduction since linear algebra can describe almost everything. Yes NNs aren't magic, but neither is the brain. Likely they use similar principles. Hinton has a theory about how real neurons might be implementing a variation of backpropagation and there are a number of other theories.
Re:AlphaGo Zero: Learning from scratch by ollin on Wednesday October 18, 2017 @11:58PM (#15505370)
>As far as we know the brain is just a "linear algebra blackbox"...Likely they use similar principles.

I'm not an expert, but my impression is that this is not really a reasonable claim, unless you're only considering very small function-like subsystems of the brain (e.g. visual cortex). Neural nets (of the nonrecurrent sort) are strict feed-forward function approximators, whereas the brain appears to be a big mess of feedback loops that is capable of (sloppily, and with much grumbling) modeling any algorithm you could want, and, importantly, adding small recursions/loops to the architecture as needed rather than a) unrolling them all into nonrecursive operations (like a feedforward net) or b) building them all into one central singly-nested loop (like an RNN).

The brain definitely seems to be using something backprop-like (in that it identifies pathways responsible for negative outcomes and penalizes them). But brains also seem to make efficiency improvements really aggressively (see: muscle memory, chunking, and other markers of proficiency), even in the absence of any external reward signal, which seems like something we don't really have a good analogue for in ANNs.

Re:AlphaGo Zero: Learning from scratch by Houshalter on Thursday October 19, 2017 @08:55AM (#15507447)
There are some parts of the brain we have no clue about. Episodic memory or our higher level ability to reason. But most of the brain is just low level pattern matching just like what NNs do.

The constraints you mention aren't deal breakers. We can make RNNs without maintaining a global state and fully unrolling the loop. See synthetic gradients for instance. NNs can do unsupervised learning as well, through things like autoencoders.

Re:AlphaGo Zero: Learning from scratch by fjsolwmv on Friday October 20, 2017 @03:39PM (#15518608)
A pattern matcher can learn high level reasoning. Reasoning is just a boolean circuit
Re:AlphaGo Zero: Learning from scratch by mannigfaltig on Wednesday October 18, 2017 @06:42PM (#15503919)
> It's an uninteresting reduction since linear algebra can describe almost everything.

The question is whether it can do so efficiently. As far as I know, alternating applications of affine transforms and non-linearities are not so useful for some computations that are known to occur in the brain such as routing, spatio-temporal clustering, frequency filtering, high-dimensional temporal states per neuron etc.

Re:AlphaGo Zero: Learning from scratch by andbberger on Wednesday October 18, 2017 @08:25PM (#15504487)
Hinton changes his opinion about what the brain is doing every 5 years... Hinton is not a neuroscientist...
Re:AlphaGo Zero: Learning from scratch by madez on Wednesday October 18, 2017 @08:46PM (#15504571)
If he changes his opinion, which I understand to be models of the brain in this case, and each iteration improves the model, then that is perfectly fine. It would be bad if someone did not change their view in case of inconsistent evidence.
Re:AlphaGo Zero: Learning from scratch by andbberger on Wednesday October 18, 2017 @08:50PM (#15504586)
For political opinions sure, but if he's changing his opinions so often ...

When you're a big scientific figure, I think that you have some extra responsibility to the public to only say things you're very confident about. Or otherwise very clearly communicate your uncertainty!!

Re:AlphaGo Zero: Learning from scratch by red75prime on Wednesday October 18, 2017 @11:24PM (#15505260)
> Hinton is not a neuroscientist...

It's not like neuroscientists know that either.