Re:Learning to think critically about machine learning
by SemanticStrengh on Monday May 02, 2022 @02:19PM
(#31238774)
I am talking about training a neural network to achieve calculations.
And yes look-up tables might be fit for addition but not for multiplication.
The accuracy would be <90% which is a joke for any serious use.
Re:Learning to think critically about machine learning
by drdeca on Tuesday May 03, 2022 @02:51AM
(#31245452)
Well, the main issue I see is where to put the n^2 memory (where n is the number of digits) when doing multiplication. (Or, doesn’t need n^2 space, could do it in less, but might need to put more structure into the architecture?)
If the weights are designed, and the network architecture allows something to hold the information needed, then there is really no obstacle to having it get multiplication entirely (not just 90%).
Now, would that be learnable? I’m not so sure, at least with the architecture one would use if designing the weights.
But,
I see no reason a transformer model couldn’t be trained on multiplication-with-work-shown and produce text fitting all of those patterns, and successfully perform multiplication for many digits that way.
And, by “showing all work” I don’t necessarily mean “in a way a person would typically show their work”, but in a easier-for-machine way.
Because, with RELU activation, I’m fairly confident that the latter, at least, is possible.
(Where inputs are given using digits (where each digit could be represented with one floating point input), and the output is also represented with digits)
Like, you can implement a lookup table with neural net architecture. That’s not an issue.
And composing a lookup table with itself a number of times lets one do addition, etc.
... ok, I suppose for multiplication you would have to like, use more working space than what would effectively be a convolution, and one might complain that this extra structure of the network is “what is really doing the work”, but, I don’t think it is more complicated than the existing NN architectures?