News for hackers. Stuff that matters.
Re:Study reveals how, when a synapse strengthens, its neighb... by alexbeloi on Sunday June 24, 2018 @06:16PM (#17388670)
My naive take on this is that it makes biologically sense to keep net expected electrical impulses approximately the same before and after strengthening. At the end of the day the brain has energy constraints.

This can-be/is done functionally in ANNs but achieves a different end (avoids over-fitting) but doesn't reducing energy(compute) expenditure in dense ANNs since activation and non-activation is computed in expectation and take the same number of cycles in dense networks.

I'd love to see more work on massive sparse networks, where you actually get compute efficiency if you can reduce number of activation without reducing hurting your optimization target.

Re:Study reveals how, when a synapse strengthens, its neighb... by im3w1l on Monday June 25, 2018 @03:18AM (#17390509)
And apart from speeding things up, it could also have a regularizing effect.
Re:Study reveals how, when a synapse strengthens, its neighb... by nabla9 on Sunday June 24, 2018 @03:12PM (#17387901)
In deep learning this is called local response normalization (LRN).

ConvNets use LRN where most active neurons inhibit other neurons at the same location in neighboring feature maps. http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf

Re:Study reveals how, when a synapse strengthens, its neighb... by adityab on Sunday June 24, 2018 @04:32PM (#17388257)
Actually the new finding is similar to weight normalisation for individual neurons. LRN would be similar to another thing called lateral inhibition.
Re:Study reveals how, when a synapse strengthens, its neighb... by jxub on Sunday June 24, 2018 @01:30PM (#17387381)
The naive conclusion is to avoid learning in any form in order to keep all the synapses in optimal state. Rather disappointing.
Re:Study reveals how, when a synapse strengthens, its neighb... by obstbraende on Sunday June 24, 2018 @01:50PM (#17387474)
That's too naive. There's next to nothing known about how activity in individual neurons and their synapses relates to mental contents, in particular "higher level" concepts and thought patterns that you're concerned with in adult learning. In other words, the network dynamics are complex and unknown, and so the suppression / deactivation of certain synapses may just as well be a normal and necessary part of learning as it may be a part of forgetting, or neither. There is currently no contender for a "neuroscience standard model" that would bridge between this kind of neural dynamics and cognitive functions. I hope to live to see one.
Re:Study reveals how, when a synapse strengthens, its neighb... by soup10 on Sunday June 24, 2018 @08:42PM (#17389193)
Amen to that, we are starting to get some of the groundwork but are far far away from a “standard model”, neuroscientists,psychiatrists and AI developers oversell their understanding in order to keep their jobs and funding.
Re:Study reveals how, when a synapse strengthens, its neighb... by SubiculumCode on Sunday June 24, 2018 @09:09PM (#17389282)
I know of no psychologist or neuroscientist who even try for a 'standard model' level of understanding of brain function.
Re:Study reveals how, when a synapse strengthens, its neighb... by haskellandchill on Sunday June 24, 2018 @11:58PM (#17389799)
It’s really just the AI folks.
Re:Study reveals how, when a synapse strengthens, its neighb... by lars on Sunday June 24, 2018 @02:57PM (#17387829)
Maximizing total synapse strength would as useful as maximizing the number of 1-bits on your hard drive. It's not what you want :)
Re:Study reveals how, when a synapse strengthens, its neighb... by drak0n1c on Sunday June 24, 2018 @02:15PM (#17387627)
Regardless of one’s general level of intelligence, there is that age-old adage of a trade-off between book smarts and street smarts, or any other niche skill. Forgetting, and the resulting ignorance, is part of learning. Perhaps that’s why people in modern societies feel unhappier than those in poorer, less developed societies.

Still, any society has certain forms of learning with outsized benefits that make them worthwhile.

Re:Study reveals how, when a synapse strengthens, its neighb... by azeirah on Sunday June 24, 2018 @02:40PM (#17387735)
I remember reading a paper a little while ago that stated that experts are better at their craft because they're better at ignoring unimportant and irrelevant input.
Re:Study reveals how, when a synapse strengthens, its neighb... by lawlessone on Monday June 25, 2018 @05:26AM (#17391016)
>Perhaps that’s why people in modern societies feel unhappier than those in poorer, less developed societies.

I have heard this before and would really question if it is true.

Has anyone studied this?

Re:Study reveals how, when a synapse strengthens, its neighb... by zerostar07 on Sunday June 24, 2018 @10:28PM (#17389545)
... and a model predicting and explaining this a decade ago: https://www.ncbi.nlm.nih.gov/pubmed/18602704
Re:Study reveals how, when a synapse strengthens, its neighb... by tunnuz on Sunday June 24, 2018 @02:06PM (#17387567)
I wonder if the progress in artificial neural network technology is going to be correlated to discoveries on real neural networks.
Re:Study reveals how, when a synapse strengthens, its neighb... by amelius on Sunday June 24, 2018 @02:08PM (#17387578)
I'm still wondering about the natural equivalent of back-propagation.
Re:Study reveals how, when a synapse strengthens, its neighb... by no_identd on Sunday June 24, 2018 @03:15PM (#17387922)
Re:Study reveals how, when a synapse strengthens, its neighb... by p1esk on Sunday June 24, 2018 @04:25PM (#17388226)
I think the term "backpropagation" has a different meaning in neurobiology.
Re:Study reveals how, when a synapse strengthens, its neighb... by zerostar07 on Sunday June 24, 2018 @10:29PM (#17389549)
those are backpropagating spikes, not errors
Re:Study reveals how, when a synapse strengthens, its neighb... by SubiculumCode on Sunday June 24, 2018 @09:13PM (#17389302)
Isn't that mostly a supervised learning technique? Maybe the larger question is whether learning in the brain is better described as unsupervised or supervised.
Re:Study reveals how, when a synapse strengthens, its neighb... by mrep on Monday June 25, 2018 @02:18AM (#17390270)
I don't know how babies learn to initially use words but after that entails 99% supervised learning through school IMO nowadays. I would even argue that most new economic growth is the also the result of general supervised learning (aka, learning from others because the wealth of human knowledge is way more than anyone can learn in a million lifetimes and so the economic growth comes from continuously combining new ideas that combine other new ideas).
Re:Study reveals how, when a synapse strengthens, its neighb... by im3w1l on Monday June 25, 2018 @03:21AM (#17390525)
But we aren't beating kids when they get facts wrong and we aren't giving them candy when they are right. So studying how the reward signal is computed should be interesting.
Re:Study reveals how, when a synapse strengthens, its neighb... by mrep on Monday June 25, 2018 @11:01AM (#17392913)
Beating kids and giving them candy is way on the extreme side of reinforcement. Having good grades be praised by teachers/parents and bad grades be associated with disappointment is already pretty good since humans are highly receptive to other people's emotions.
Re:Study reveals how, when a synapse strengthens, its neighb... by SubiculumCode on Tuesday June 26, 2018 @04:28PM (#17403735)
Really we are being too static in the thinking about it. Supervised learning can essentially be done by one brain region supervising another that is being trained. It doesn't have to be outside the skull.
Re:Study reveals how, when a synapse strengthens, its neighb... by im3w1l on Tuesday June 26, 2018 @01:23AM (#17398055)
Yeah I agree! I'm just saying there's gotta be a lot of intricate machinery to make such subtle reinforcements work.
Re:Study reveals how, when a synapse strengthens, its neighb... by raincom on Sunday June 24, 2018 @01:34PM (#17387398)
Re:Study reveals how, when a synapse strengthens, its neighb... by manmal on Monday June 25, 2018 @01:13AM (#17390026)
So this is how Eg brainwashing works? Repeating a message strengthens the new pathways until the old connections are too weak to fire?
Re:Study reveals how, when a synapse strengthens, its neighb... by evincarofautumn on Monday June 25, 2018 @02:11AM (#17390237)
I would think any kind of training would have this effect. In mechanism—not purpose or ethics—brainwashing doesn’t strike me as all that different from ordinary practice.
Re:Study reveals how, when a synapse strengthens, its neighb... by leeeeech on Monday June 25, 2018 @03:54AM (#17390664)
Arguably, a single message would only affect a single pathway.
Re:Study reveals how, when a synapse strengthens, its neighb... by manmal on Monday June 25, 2018 @05:05AM (#17390937)
AFAIK a single message causes a giant electrical and chemical thunderstorm in the brain, not only one excited single pathway?
Re:Study reveals how, when a synapse strengthens, its neighb... by leeeeech on Tuesday June 26, 2018 @05:05AM (#17398963)
You are right of course. But because a trivial message has many "layers" of "messages", according to semantic theory, it's often a single lie in a whole message packet.
Re:Study reveals how, when a synapse strengthens, its neighb... by pmalynin on Sunday June 24, 2018 @02:51PM (#17387788)
So basically L2 regularization.
Re:Study reveals how, when a synapse strengthens, its neighb... by p1esk on Sunday June 24, 2018 @04:23PM (#17388223)
Not really. This is a local inhibition, and it's not clear if there's penalty on synapse growth.
Re:Study reveals how, when a synapse strengthens, its neighb... by ridgeguy on Sunday June 24, 2018 @09:57PM (#17389446)
How interesting.

This looks like a lateral inhibition effect running on biochemical means that are persistent compared to the transient electrochemical phenomena of classic neuronal lateral inhibition. The latter does lots of signal processing in visual and other systems. [1]

[1] https://en.wikipedia.org/wiki/Lateral_inhibition

Re:Study reveals how, when a synapse strengthens, its neighb... by audiolion on Sunday June 24, 2018 @02:03PM (#17387550)
We are born with a ton of synapses as babies and through a process called pruning, many synapses die off, unused branches.

This seems pretty normal to me

Re:Study reveals how, when a synapse strengthens, its neighb... by jchook on Sunday June 24, 2018 @01:43PM (#17387442)
Really curious what the biology-centric AI folks have to say about this, e.g. Jeff Hawkins.
Re:Study reveals how, when a synapse strengthens, its neighb... by rytill on Sunday June 24, 2018 @02:00PM (#17387530)
Normalization of a layer’s outputs kind of does this already.
Re:Study reveals how, when a synapse strengthens, its neighb... by p1esk on Sunday June 24, 2018 @04:15PM (#17388194)
So it's similar to "lateral inhibition", but for synapses.
Re:Study reveals how, when a synapse strengthens, its neighb... by bsenftner on Sunday June 24, 2018 @02:07PM (#17387573)
Isn't that kinda the point? It's called learning.
Re:Study reveals how, when a synapse strengthens, its neighb... by techsin101 on Sunday June 24, 2018 @09:27PM (#17389359)
Does this mean addiction can completely be cured
Re:Study reveals how, when a synapse strengthens, its neighb... by agumonkey on Sunday June 24, 2018 @02:40PM (#17387734)
self pruning mechanism ?