Richard Hamming has a story where he was shocked/appalled that some folks wanted to include him on a paper after he suggested using Linear Regression in an obvious application of linear regression. The upshot is, if it were obvious it would have been done already. So many times people have done everything they can to avoid using a matrix decomposition or a proper filter.

A lot of expertise is really:

1. Knowing when to use the right tool, and

2. Being in the right place at the right time to offer the insight.

What’s obvious to the expert is frequently intractable to the outsider. In many cases, I wouldn’t call it so much “avoidance” as “not knowing the solution exists at all”.

In my area, every controls engineer says "we can just use a PID".

99% of the time it seems they're not wrong, despite control engineering being a pretty large area of research.

I'm nominally familiar with PID loops but am not a control engineer. What other tools would you commonly bring to bear when a PID is not applicable?

And a simple PID controller built with op-amps looks an awful lot like a simple neural network.

http://www.ecircuitcenter.com/Circuits/op_pid/op_pid.htm

https://www.nutsvolts.com/magazine/article/the_perceptron_ci...

Isn't gradient descent basically PID over parameters? And tricks like momentum basically a low-pass filter integrated in the PID loop? It's quite weird how not that many concepts from analog electronics domain have gotten carried over to ML.

I mean yes and no? Kalman filters fuses information about known system behaviour along with observations in an optimal predictor (for certain types of noise). It uses a matrix in the same way that almost every operation on a computer uses a matrix.

Edit: I missed the quip about the Kalman comment being in jest, my reply probably wasn’t necessary but I’ll leave it up.