Re:Gzip and KNN Outperforms Transformers on Text Classification
by lmeyerov on Thursday July 13, 2023 @10:34AM
(#36709776)
Even just undergraduate linear algebra & calc + real analysis & prob/stats... With good teachers to draw the connections
Stuff like info theory are amazing bc of this... But easy to miss if you are just working through a drier text
The crazier version for me is we have a physics professor on our team whose grounding intuitions are use ideas like black holes for mental intuition... Which does not work as well for the rest of the team
(And a lot of modern ML/AI feels very engineered and interchangeable after that, like 'metric function of the month')
Stuff like info theory are amazing bc of this... But easy to miss if you are just working through a drier text
The crazier version for me is we have a physics professor on our team whose grounding intuitions are use ideas like black holes for mental intuition... Which does not work as well for the rest of the team
(And a lot of modern ML/AI feels very engineered and interchangeable after that, like 'metric function of the month')