Re:Large Language Models' Emergent Abilities Are a Mirage
by CamperBob2 on Monday March 25, 2024 @12:25PM
(#39818257)
I don't know... the ability to write code to solve an otherwise ill-suited problem seems pretty general to me. It seems like a big step in a concrete direction, as opposed to a lot of Goedelian navel-gazing about arithmetic and Peano axioms and whatnot.
Agreed that generalized architectures will ultimately win out over hand-tweaked ones. But the patent wars that will eventually be fought over this stuff are where the real bitter lessons will come into play. At some point, we'll be forced back into the hand-optimization business because someone like OpenAI (or another Microsoft proxy) will have locked down a lot of powerful basic techniques.
Agreed that generalized architectures will ultimately win out over hand-tweaked ones. But the patent wars that will eventually be fought over this stuff are where the real bitter lessons will come into play. At some point, we'll be forced back into the hand-optimization business because someone like OpenAI (or another Microsoft proxy) will have locked down a lot of powerful basic techniques.