applecoffeecake
applecoffeecake on Hacker News
The answer is that it just takes continuous and serious engagement with both general and specific mathematics typically through graduate-level courses and research degrees. The particular interpretation of least squares as noise whitening plus orthogonal projection is just a trick you would learn in an advanced statistics class. If you are not versed in the subject, you would not immediately understand the particular math here even as a mathematician! To demonstrate this point, at 56:00 in this video (
https://youtu.be/VcUU42ZXnQs?si=kqkUWxnFv2lSa5FH), Fields Medalist Cedric Villani is confused by a similar remark about noise whitening and asks the speaker to explain what it means. Here, the speaker is applying the trick to the closely related concept of Linear Minimum Mean Squared Error (LMMSE) estimation which is the probability-theoretic counterpart to least squares. Moreover, Cauchy-Schwarz refers to a fundamental inequality that any mathematician would know, and would show the optimality of orthogonal projection. The general (Cauchy-Schwarz) gives you a foundation for understanding the specific (least squares theory).
I switched to a Julia + C workflow from a MATLAB + Python + C workflow early in my graduate school years. In my view, Julia is the only real alternative to MATLAB if you do any kind of numerical, scientific, or technical computing. People like to mock MATLAB but MATLAB is both faster and significantly less verbose than Python.
The worst thing about Python is the loop phobia and emphasis on vectorization and slick one-liners. There's more to numerical algorithms than matmuls and pointwise nonlinearities. The problem is not only that most nontrivial algorithms don't vectorize easily if at all, but that nothing intellectually is gained by forcing the user to waste energy contorting what they already understand as a for loop into a slick one-liner only for that one-liner to be implemented under the hood as the loop that the user had in mind in the first place.
There are two further ironies here. The first is that writing something as a loop is typically the first step towards understanding and controlling dependencies and parallelism. The second is that parallel hardware is itself described in HDL by spatial for loops with explicit indexing, because indices are the clearest way to express how stages connect, which is exactly what the programmer has in mind in the first place.
Tao is clearly referring to mathematics journals. Any kind of practically useful result can easily be published in an engineering or applied scientific journal.
If a result has a real-world application, then it can easily be published in an engineering or applied scientific journal in which it is already the norm to present methods that work empirically with little to no understanding of how.