Re:The day I taught AI to read code like a Senior Developer
by jprete on Sunday January 05, 2025 @10:26AM
(#42602378)
The Bitter Lesson claimed that the best approach was to go with more and more data to make the model more and more generally capable, rather than adding human-comprehensible structure to the model. But a lot of LLM applications seem to add missing domain structure until the LLM does what is wanted.
[ Parent
| on HN ]
Re:The day I taught AI to read code like a Senior Developer
by Philpax on Sunday January 05, 2025 @12:11PM
(#42603191)
The Bitter Lesson states that you can overcome the weakness of your current model by baking priors in (i.e. specific traits about the problem, as is done here), but you will get better long-term results by having the model learn the priors itself.
That seems to have been the case: compare the tricks people had to do with GPT-3 to how Claude Sonnet 3.6 performs today.
Re:The day I taught AI to read code like a Senior Developer
by do_not_redeem on Sunday January 05, 2025 @10:37AM
(#42602467)
The Bitter Lesson pertains to the long term. Even if it holds, it may take decades to be proven correct in this case. Short-term, imparting some human intuition is letting us get more useful results faster than waiting around for "enough" computation/data.