News for hackers. Stuff that matters.

A comment on The Computational Power of Biological Dendritic Trees. Parent comment

Re:The Computational Power of Biological Dendritic Trees by slx26 on Tuesday September 08, 2020 @11:31AM (#24409390)
I don't think they understimate the non-linearity of real neurons. It's simply a trade-off. Accurate models were unpractical (too computationally expensive) the last time I checked. There's quite a bit of research on neuron modelling, both for accurate responses and approximations that are fast to compute. The wikipedia article [0] is actually a really good read. What's true is that with the current upscaling of neural networks, it might be worth it to look back at more sophisticate approximations and see if they are implementable now. But I guess the problem would not be just training, but the fact that the trained networks would also be more expensive to run. I don't really know much about the topic, so if anyone has studied this in more detail and wants to share something...

[0] https://en.wikipedia.org/wiki/Biological_neuron_model

[ Parent | on HN ]
Re:The Computational Power of Biological Dendritic Trees by a-priori on Tuesday September 08, 2020 @06:06PM (#24414090)
It's tricky to be both reasonably biologically accurate, and yet computationally efficient. Some models do exist to fit this niche. One I'm familiar with is Izhikevich (2003), a model with four morphology parameters and two state variables and corresponding ordinary differential equations. It can simulate the spiking behaviour of several different kinds of neurons.

https://www.izhikevich.org/publications/spikes.htm

I've written a SIMD-based simulator that can integrate 500k of those neurons in real-time on a standard laptop. That's a bit of a limit case -- it doesn't yet model action potentials or learning, which would slow it down quite a bit. The details here matter a lot: a GPU version I experimented with could do about 1M, whereas an earlier version of this work that was a straight-up CPU-based solution (_not_ SIMD) but _with_ action potentials and learning could only model 1k.

[ Parent | on HN ]
Re:The Computational Power of Biological Dendritic Trees by slx26 on Wednesday September 09, 2020 @01:36PM (#24423525)
That's very interesting, thanks for sharing!
[ Parent | on HN ]