News for hackers. Stuff that matters.
Re:A Google engineer who thinks the company’s AI has come to... by Thiez on Sunday June 12, 2022 @03:24AM (#31712375)
The brain is a pretty messy place. While our bodies try hard to preserve homeostasis, conditions will always vary slightly. Hormones affect our brains and can vary in concentration from moment to moment. Type 1 diabetics can easily have their blood sugar increase by a factor of 4, without losing consciousness. There are many other chemicals and conditions (temperature?) that similarly influence the condinions under which our neurons must operate.

But even in this noisy environment, our brains can usually operate just fine. Given this level of background noise, I would be very surprised if the average synapse would require more than a 16-bit floating point value for its outputs.

Re:A Google engineer who thinks the company’s AI has come to... by skohan on Sunday June 12, 2022 @03:51AM (#31712525)
Counter argument:

Some synapses contain multiple receptor sub-types. Dopamine, for example, has some receptors which are inhibitory, and others which are excitatory. Each receptor is sensitive to different concentrations of dopamine. This gives the effect that when the dopamine concentrations in the synapse are low, this synapse acts to inhibit the activity of the efferent neuron, while when the concentration reaches a certain threshold, the output of the synapse flips to an excitatory signal.

How would you model that system with a single F16?

And that's just one example: for instance there are specific signals which trigger cascading effects in the neuron which up-regulate certain chemical pathways, causing an amplifying effect on other signals.

Or you have shunting effects, where an inhibitory synapse placed at a certain point in the dendritic arbor serves to selectively cancel signals from farther down that one specific branch of the dendrite.

That's not "noise" - the brain has incredibly refined and detailed mechanisms for information processing which go very far beyond `weight * sigmoid(activation) = output`