As far as I know, we're closest to showing information processing in the visual cortex (which is highly linear) and we're still a long way from knowing how it works at a neural level. But maybe someone here can update on this?
But much of the cortex is highly recurrent (non-linear) and the idea that it's doing something like sending bits between synapses, encoded in spike timing or something.. well, I think that's highly speculative and has plenty of problems. But even if so, that's just "information processing".
I'm personally a fan of electromagnetic theories of consciousness[], where the synaptic activity could be an epiphenomenon of supporting a stand EM field.
[]https://en.wikipedia.org/wiki/Electromagnetic_theories_of_co...
I am not sure how much is known about information processing, but it's clear that motor impulses and sensory information are encoded in the spikes. Higher spike frequency = stronger signal. Synapses are how signals are passed from neuron to neuron.
Afaik, we have correlative descriptions of what these waves indicate (importantly, they're associated with sleep, consciousness, attentiveness), but no direct mechanical model of them or a clear purpose. So yeah, spike timing could still be used at this level, but it seems other behaviors are also happening that may be more essential to the larger function.
The sentiment that synapses probably don't explain everything is rather common, anyhow. I'm thinking, the way the blood flow literally influences the relevant areas by transporting available energy for example, and neurotransmitters must be a very important point, and how those areas react in case of insufficiency would explain why I become nasty when tired and hungry at the same time.
See more: https://en.wikipedia.org/wiki/Neural_correlates_of_conscious...
Training one of these big models takes 100kWh for 1e19 flops, so that's 100k Wh, 360M Ws, or 360MJ or 3.6 1e8J. 1e8Joules/1e19flops = 1e-11J/flop
Neurons take 1e-8J/spike.[1]
Math check appreciated :)
Does seem plausible to think of a single neuron spike (hodgkin-huxley cable model) being modeled with ~1k flops. Though I'm firmly of the opinion that nobody really knows how the brain works.. the neural spike activity could be pure epiphenomenon.. who knows!
[1] “Finally, the energy supply to a neuron by ATP is 8.31 × 10−9 J. Meanwhile, integrating the total power with respect to time we will get the consumed electric power, which is 8.75 × 10−9 J. This is more energy than the ATP supplied. The energy efficiency is 105.3%. This is an anomaly…” - 2017 Feb 16 Wang, Xu, Institute for Cognitive Neurodynamics, East China University of Science and Technology https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5337805/