FrereKhan

This paper imports an arbitrarily-chosen aspect of cortical architecture — topological maps of function — and ignores every other aspect of biological neural tissue. The resulting models show lower performance for the same number of parameters — not surprising, since they are more constrained compared with baseline. They may be slightly more robust against pruning — not surprising, since they are more regularised.

The figures show individual seeds, presumably, with no statistical analysis in the performance or pruning comparisons, so the null hypothesis is there is no difference between toponets and baseline. I would never let this paper be submitted by my team.

We haven't learned anything about the brain, or about ANNs.

I agree. The use case needs to justify having state, otherwise the ideal architecture is something like NullHop. Temporal signal processing / vision processing tasks are ideal for SNNs, especially if the inputs can also be sparse.

Ultra-low-power neuromorphic processors such as DynapSE[1] have been cross-bar free for several years now, making them a perfect fit for sparse networks (both weight- and activity-sparsity). [1] https://arxiv.org/abs/1708.04198

If you bring activation sparsity into the mix, the advantage of SNN processors over GPUs/TPUs becomes more clear. Loss-gradient-based optimisation approaches are great because they give you a tool to include e.g. sparsity regularisation into the loss. Encouraging sparse activity makes simple linear algebra a poor fit for network activation, and SNN processors a much better fit.

It's not quite correct to say this is only for achieving deep learning. Gradient-based parameter optimisation is still a useful tool, even for small shallow networks that would be ideal for event-based signal processing.

Even for small-network tasks, training spiking networks has been non-trivial. This paper provides a way to get exact gradients, implying probably faster optimisation than using surrogate gradients or other approximation methods for SNNs.

aiCTX (ai-ctx.com) | Real-time low-power machine learning | SW Eng., Silicon HW Eng., ML research | Zürich, Switzerland | ONSITE, INTERNS, FULL-TIME

http://bit.ly/aiCTX_jobs1

aiCTX is a small (<10 employees) startup based in Switzerland, with a diverse group of SW, HW and ML engineers. We design ultra low power analog and mixed-signal neuromorphic silicon, built for real-time machine learning applications.

We're looking for awesome people to help build our SW stack; to help our HW designers for the next round of chips; and to help our research team build solutions for particular application domains.

Zürich is cool, and Switzerland is beautiful. :)

Check out the position details in the link above if you'd like to join us!