News for hackers. Stuff that matters.
Re:Things we learned about LLMs in 2024 by jsheard on Tuesday December 31, 2024 @03:30PM (#42561597)
I swear these goalposts keep getting moved, I remember being told that GPT3.5 is a useless toy but the paid GPT4 is lifechanging, and now that GPT4 is free I'm told that it's a useless toy but paid o1 or paid Sonnet are lifechanging. Looking forward to o1 and Sonnet becoming useless toys, unlike the lifechanging o3.
Re:Things we learned about LLMs in 2024 by aetherson on Tuesday December 31, 2024 @03:47PM (#42561712)
You will also be dismayed to hear that a 2011 iPhone is no longer state-of-the-art, and indeed can't run most modern apps.
Re:Things we learned about LLMs in 2024 by jpc0 on Tuesday December 31, 2024 @04:22PM (#42561914)
GPT4 is a 13 year old technology? Compared to o1 and Sonnet 3.5?

If someone told me an iPhone 4 is terrible but an iPhone 5 would definitely serve my needs, then when I get an iPhone 5 they say the same of the 6 you really want me to believe them a second time? Then a third time? Then a 4th? In the mean time my time and money is wasted?

Re:Things we learned about LLMs in 2024 by scubbo on Tuesday December 31, 2024 @04:23PM (#42561925)
Holy false-equivalency, Batman! The definitions of "useless toy / lifechanging tool" are _not_ changing over time (or, at least, not over the timescale being explored here), whereas the expectations and requirements of processing power of a phone are.
Re:Things we learned about LLMs in 2024 by aetherson on Wednesday January 01, 2025 @11:52AM (#42567205)
But in fact they are changing over time -- this is an expectations treadmill. When you get something newer and better, it highlights the flaws in what you had before.
Re:Things we learned about LLMs in 2024 by scubbo on Wednesday January 01, 2025 @06:04PM (#42570044)
That is true _in general_, but not in this specific case (hence why I specified "not over the timescale being explored here"). A modern cigarette-lighter would indeed have been a life-changing tool to a caveman but is indeed disposable junk today.

The point being made by the original comment (with which I agree) was that many criteria-for-usefulness - primarily that of reliability or a lack of hallucination - have remained static; with successive generations of tools being (falsely) claimed to meet them, but then abandoned when the next hype-train comes along.

I certainly agree that _some_ aspects of AI models are indeed improving (often drastically!) over time (speed, price, supported formats, history/context, etc.) - but they still _all_ fall _drastically_ short on the key core requirement that is required in order to make them Actually Useful. "X is better than Y" does not imply "where Y failed to be useful, X now succeeds".

Re:Things we learned about LLMs in 2024 by johnrob on Tuesday December 31, 2024 @06:35PM (#42562677)
It would be quite useful if that were the only phone available.
Re:Things we learned about LLMs in 2024 by raincole on Tuesday December 31, 2024 @10:14PM (#42563671)
Except GPT4 isn't free.

The GP is claiming GPT4o is bad but Sonnet is good. GPT4o is about only 20% cheaper than Sonnet.