Re:Test-Driven Development with an LLM for Fun and Profit
by sdesol on Thursday January 16, 2025 @06:35PM
(#42732332)
Well given the price $15.00 / 1M input tokens and $60.00 / 1M output* tokens, I would hope so. Given the price, I think it is fair to say it is doing a lot of checks in the background.
Re:Test-Driven Development with an LLM for Fun and Profit
by energy123 on Thursday January 16, 2025 @06:51PM
(#42732460)
It is expensive. But if I'm correct about o1, it means user mistrust of LLMs is going to be a short-lived thing as costs come down and more people use o1 (or better) models as their daily driver.
Re:Test-Driven Development with an LLM for Fun and Profit
by sdesol on Thursday January 16, 2025 @07:33PM
(#42732737)
> mistrust of LLMs is going to be a short-lived thing as costs come down and more people use o1
I think the biggest question is, is o1 scalable. I think o1 does well because it is going back and forth hundreds if not thousands of times. Somebody mentioned in a thread that I was participating in that they let o1 crunch things for 10 minutes. It sounded like it saved them a lot work, so it was well worth it.
Whether or not o1 is practical for the general public is something we will have to wait and see.
Re:Test-Driven Development with an LLM for Fun and Profit
by energy123 on Thursday January 16, 2025 @07:48PM
(#42732826)
I'm going to wager "yes" because o3-mini (High) gets equal benchmark scores to o1 despite using 1/3rd as much compute, and because the consistent trend has been towards rapid order-of-magnitude decreases in price for a fixed level of intelligence (trend has many components dovetailing, both hardware and software related). Can't forecast the future, but this would be my bet on a time horizon of < 3 years.
My experience is, o1 is very good at avoiding hallucinations and I trust it more, but o1-mini and 4o are awful.