Reservoir computing approaches usually have the intermediate layers be recurrent, eg they implement a dynamical system. Theoretically, this is actually Turing complete, although good luck programming it. But in any case, the range of behaviors — transformations in the data — that a dynamical system can implement is much more powerful than just a basis expansion. However, whether this is what is happening in actual practice is really, really unclear to me. A lot of recurrent neural networks aren't doing anything more than an equivalent feedforward network, and the same thing may be true here: reservoir approaches might, for the most part, really just be performing nonlinear projection to higher dimensional spaces, and then the output layer is being trained to classify those patterns.
Re:Reservoir computing
by gbrown on Thursday October 18, 2018 @10:40PM
(#18253915)
So treat it like a basis expansion?
[ Parent
| on HN ]
Re:Reservoir computing
by GlenTheMachine on Thursday October 18, 2018 @10:49PM
(#18253944)
If I understand the question correctly, ie is a reservoir computing approach simply projecting an input vector into a higher dimensional space, like a wonky support vector machine, then I think the answer is: unclear.