News for hackers. Stuff that matters.
Re:You could have invented transformers from memory by applecoffeecake on Tuesday October 20, 2026 @05:47PM (#50173147)
The answer is that it just takes continuous and serious engagement with both general and specific mathematics typically through graduate-level courses and research degrees. The particular interpretation of least squares as noise whitening plus orthogonal projection is just a trick you would learn in an advanced statistics class. If you are not versed in the subject, you would not immediately understand the particular math here even as a mathematician! To demonstrate this point, at 56:00 in this video (https://youtu.be/VcUU42ZXnQs?si=kqkUWxnFv2lSa5FH), Fields Medalist Cedric Villani is confused by a similar remark about noise whitening and asks the speaker to explain what it means. Here, the speaker is applying the trick to the closely related concept of Linear Minimum Mean Squared Error (LMMSE) estimation which is the probability-theoretic counterpart to least squares. Moreover, Cauchy-Schwarz refers to a fundamental inequality that any mathematician would know, and would show the optimality of orthogonal projection. The general (Cauchy-Schwarz) gives you a foundation for understanding the specific (least squares theory).

There are no comments on this comment yet.