Re:An Analog Network of Resistors Promises Machine Learning ...
by mistercow on Sunday June 30, 2024 @11:30AM
(#40837741)
There is a separate interesting question of whether you could exploit the nonlinearity of a network of non-ideal resistors in practice. Given that tom7 was able to use floating point inaccuracy as a source of nonlinearity for ML, I’m guessing the answer is probably “yes, but now your system is incredibly sensitive to stuff like ambient temperature”.
Re:An Analog Network of Resistors Promises Machine Learning ...
by duped on Sunday June 30, 2024 @11:20AM
(#40837685)
By definition it can't. An ideal resistor is a linear relation between the voltage across it and the current through it.
If you are allowed switches in the network and the input is a fixed voltage then things get interesting.
Re:An Analog Network of Resistors Promises Machine Learning ...
by klyrs on Sunday June 30, 2024 @07:13PM
(#40841191)
However, in a real resistor, resistivity varies with temperature, and current through the resistor produces heat. Therefore, real resistors are in fact nonlinear circuit devices. Technically speaking. In reality, whenever electrical engineers are confronted with this nonlinearity, it's a bug that must be smashed and not a feature to exploit.
Re:An Analog Network of Resistors Promises Machine Learning ...
by JKCalhoun on Sunday June 30, 2024 @10:44AM
(#40837424)
Just spitballing: but I'm thinking about how slide rules are linear but can do calculations that, I believe?, are not limited to being linear. Due I imagine to logarithmic rulings and that logarithms can be added/subtracted in a linear fashion.
Re:An Analog Network of Resistors Promises Machine Learning ...
by utensil4778 on Sunday June 30, 2024 @12:09PM
(#40837993)
That's more or less what we have now. Slide rules and computers are linear systems which can compute nonlinear functions. The problem is that computing anything takes time and must be a linear set of instructions executed in series (multiplied by many parallel cores).
Using an analog approach could be vastly more efficient as operations are inherently parallel. You can fire off every neuron in a layer simultaneously and produce a result within nanoseconds, for basically any number of neurons. You could probably do the entire network as a single atomic operation, but that's a bit beyond my knowledge of neural networks
But seriously, my actual question is whether an ideal resistor network can compute nonlinear functions since the individual resistors are linear in their input, ignoring possible nonlinear effects like temperature dependence of the conductivity of the resistors