News for hackers. Stuff that matters.
Re:An Analog Network of Resistors Promises Machine Learning ... by rsfern on Sunday June 30, 2024 @08:22AM (#40836769)
This is really cool, but I was confused by the framing as a resistor network since I think that should be linear (to first order? I’m not an EE)

What they have is a transistor network, and they constrain all the transistors to the ohmic regime, so now the resistivity of an individual transistor can be some nonlinear function of its inputs, which is really cool, like detuning transistors to do analog computation instead of digital.

Here’s the preprint: https://arxiv.org/abs/2311.00537

Re:An Analog Network of Resistors Promises Machine Learning ... by arbuge on Sunday June 30, 2024 @11:42AM (#40837827)
Once the training is complete, one thing I didn't see mentioned in the paper was how they maintain the charge on the gate capacitors, which is analogous to the weights in a traditional neural network if I'm understanding this correctly. Any practical implementation will need to have some practical way to refresh that on a continuous basis so that the weights don't drift. Was this perhaps mentioned somewhere and I missed it?
Re:An Analog Network of Resistors Promises Machine Learning ... by trueismywork on Sunday June 30, 2024 @04:47PM (#40840245)
One can use MOS capacitors for this which essentially double up as flash drives so have stprage and refresh built in. Just like flash drives, you can only do limited amount of training on them.
Re:An Analog Network of Resistors Promises Machine Learning ... by RobotToaster on Sunday June 30, 2024 @11:13AM (#40837634)
Interesting, I wonder if this can be used to create a programmable BEAM nervous network https://en.wikipedia.org/wiki/BEAM_robotics
Re:An Analog Network of Resistors Promises Machine Learning ... by utensil4778 on Sunday June 30, 2024 @10:41AM (#40837409)
Yeah, a purely resistive network will be linear in all respects (discounting thermal and related drift).

A network of transistors operating below saturation makes much, much more sense. It's really directly analogous to how we compute neuron activation in software, but inherently massively parallel.

Re:An Analog Network of Resistors Promises Machine Learning ... by pfdietz on Sunday June 30, 2024 @10:06AM (#40837232)
> linear (to first order?

I love it when things are linear to first order.

>.>

Re:An Analog Network of Resistors Promises Machine Learning ... by rsfern on Sunday June 30, 2024 @10:35AM (#40837371)
Doesn’t everyone?

But seriously, my actual question is whether an ideal resistor network can compute nonlinear functions since the individual resistors are linear in their input, ignoring possible nonlinear effects like temperature dependence of the conductivity of the resistors

Re:An Analog Network of Resistors Promises Machine Learning ... by mistercow on Sunday June 30, 2024 @11:30AM (#40837741)
There is a separate interesting question of whether you could exploit the nonlinearity of a network of non-ideal resistors in practice. Given that tom7 was able to use floating point inaccuracy as a source of nonlinearity for ML, I’m guessing the answer is probably “yes, but now your system is incredibly sensitive to stuff like ambient temperature”.
Re:An Analog Network of Resistors Promises Machine Learning ... by duped on Sunday June 30, 2024 @11:20AM (#40837685)
By definition it can't. An ideal resistor is a linear relation between the voltage across it and the current through it.

If you are allowed switches in the network and the input is a fixed voltage then things get interesting.

Re:An Analog Network of Resistors Promises Machine Learning ... by klyrs on Sunday June 30, 2024 @07:13PM (#40841191)
However, in a real resistor, resistivity varies with temperature, and current through the resistor produces heat. Therefore, real resistors are in fact nonlinear circuit devices. Technically speaking. In reality, whenever electrical engineers are confronted with this nonlinearity, it's a bug that must be smashed and not a feature to exploit.
Re:An Analog Network of Resistors Promises Machine Learning ... by JKCalhoun on Sunday June 30, 2024 @10:44AM (#40837424)
Just spitballing: but I'm thinking about how slide rules are linear but can do calculations that, I believe?, are not limited to being linear. Due I imagine to logarithmic rulings and that logarithms can be added/subtracted in a linear fashion.
Re:An Analog Network of Resistors Promises Machine Learning ... by utensil4778 on Sunday June 30, 2024 @12:09PM (#40837993)
That's more or less what we have now. Slide rules and computers are linear systems which can compute nonlinear functions. The problem is that computing anything takes time and must be a linear set of instructions executed in series (multiplied by many parallel cores).

Using an analog approach could be vastly more efficient as operations are inherently parallel. You can fire off every neuron in a layer simultaneously and produce a result within nanoseconds, for basically any number of neurons. You could probably do the entire network as a single atomic operation, but that's a bit beyond my knowledge of neural networks

Re:An Analog Network of Resistors Promises Machine Learning ... by fsckboy on Sunday June 30, 2024 @02:33PM (#40839038)
>the framing as a resistor network since I think that should be linear

transistors are essentially non-linear, but they amplify and can be made to amplify linearly through the use of feedback resistors: if you divide the output voltage across a resistor pair which fixes the output as ratio to the input voltage, that geometric relationship will hold across broad range of input/outputs. for most applications you want linear amplification. (transistors work as a function of current, but passing the current through resistors yields a voltage measurement) a transistor can be thought of as resistive if you treat its voltage:current relationship as a measurement of resistance.

Re:An Analog Network of Resistors Promises Machine Learning ... by fastglass on Sunday June 30, 2024 @04:34PM (#40840161)
essentially when there's an analog, or non-digital change in resistance over time then you can have non-linear function over that circuit
Re:An Analog Network of Resistors Promises Machine Learning ... by fastglass on Sunday June 30, 2024 @05:15PM (#40840444)
essentially when there's an analog, or non-computational change in resistance, like a drift in value, then you can have non-linear function over that circuit
Re:An Analog Network of Resistors Promises Machine Learning ... by inhumantsar on Monday July 01, 2024 @12:39AM (#40842572)
so, a new type of memristor?