GPT-5: Key characteristics, pricing and system card

Re:GPT-5: Key characteristics, pricing and system card by wongarsu on Thursday August 07, 2025 @04:06PM (#44829697)

If cost is no concern (as in infrequent one-off tasks) then you can always go with the biggest model with the most reasoning. Maybe compare it with the biggest model with no/less reasoning, since sometimes reasoning can hurt (just as with humans overthinking something).

If you have a task you do frequently you need some kind of benchmark. Which might just be comparing how good the output of the smaller models holds up to the output of the bigger model, if you don't know the ground truth

[ Parent | on HN ]

Re:GPT-5: Key characteristics, pricing and system card by Breza on Thursday August 14, 2025 @11:51PM (#44908438)

I agree. Public benchmarks aren't very useful for a bunch of reasons. Any company relying on LLMs for a critical function should have its own internal benchmark system. I maintain such a system for my job. If you are able, use the same prompt every time. It's fun to be able to include models like the original Bard on our leader board.

[ Parent | on HN ]