Re:'Memtransistor' Forms Foundational Circuit Element to Neu...
by jimrandomh on Monday February 26, 2018 @09:53PM
(#16470768)
All of the MNIST accuracy scores mentioned in these links are terrible, even by the standards of untuned novelty algorithms.
Re:'Memtransistor' Forms Foundational Circuit Element to Neu...
by cs702 on Tuesday February 27, 2018 @08:37AM
(#16473179)
Of course, because the nets we're talking about are really, really tiny.
As you can see for yourself in figure [1]c, the memristor net used in that paper has only 15 units (yes, fifteen): five memristors in the "reservoir" and 10 output neurons. The entire net is one linear transformation followed by a SoftMax, with a total of 5 × 10 = 50 parameters.
It's an impressive result that only hints at the things that will be possible when we have nets with millions or billions of memristor units.
[1] https://www.nature.com/articles/s41467-017-02337-y/figures/1
Re:'Memtransistor' Forms Foundational Circuit Element to Neu...
by jimrandomh on Tuesday February 27, 2018 @06:24PM
(#16479116)
I don't think it says anything about what a million or billion memristor network would be able to do. I don't think they scale in the same way normal machine learning algorithms do.
The big deal about memristors for AI is that they have memory and therefore produce different readout outputs for different input sequences over time -- out-of-the-box, without requiring any training -- and in a way that can make sequences linearly separable under fairly general conditions. For instance, in the Nature paper I linked to above, the researchers took a network of memristors, which they call the "reservoir," added a linear layer with a SoftMax on top of the reservoir, trained this hybrid network on a lower-resolution variant of MNIST (feeding pixel values over time, as varying voltages), and achieved classification accuracy superior to a tiny neural net despite having only 1/90th the number of neurons.[1] Note that they only trained the added layer; they did not have to train the reservoir. Figure (a) in this image has a simplified diagram of the reservoir + added layer architecture: https://www.nature.com/articles/s41467-017-02337-y/figures/1 -- only the matrix Θ had to be learned. The potential, over time, is for having highly scalable hardware neural-net components for learning to recognize and work with sequences.
PS. For clarity's sake, I'm ignoring a lot of important details and playing fast and loose with language. If you're really curious about this, please read the IEEE article and the Nature paper.
[1] https://news.engin.umich.edu/2017/12/new-quick-learning-neur...