Re:Ask HN: How to get back into AI?
by PartiallyTyped on Saturday December 10, 2022 @02:13PM
(#33935755)
Sure enough LLMs are not going to win every kaggle competition. But... I am fairly certain that transformers may. Embed all categorical values, scale continuous features by embeddings, and run it through a graph neural network, with high probability it will beat nearly everything.
Transformers require a lot of data to converge. There’s a reason tree models are still king of kaggle even though transformers have been around for 5 years now.
To me, the vast majority of people in the field seem to hold on to the idea that different technology suits different problems.
Also, are you aware that one of the most prominent AI tools of this month (ChatGPT) was obtained with RL!
Yes, LLMs are amazing but they won't be winning every single Kaggle competition, displacing every other ML algorithms in every setting.