OpenAI's new "Orion" model reportedly shows small gains over GPT-4

Re:OpenAI's new "Orion" model reportedly shows small gains o... by whywhywhywhy on Monday November 11, 2024 @06:39AM (#42106424)

I just don't take any of the testing of these seriously anymore, OG GPT4 before the "GPT Store" and "Dev Day" this time last year still just felt way more powerful than anything on the market today.

3.5 Sonnet felt close but even that feels like it's being downgraded now.

Just feels like things are silently being quantized or downgraded in other ways to increase profitability once people jump on. Even trying same prompts that one shotted flawlessly pre-Dev Day 2023 now just fail for me on GPT.

[ Parent | on HN ]

Re:OpenAI's new "Orion" model reportedly shows small gains o... by Wheatman on Monday November 11, 2024 @06:48AM (#42106478)

This seems to be a prevalent problem that people bring up.

I remember there was a forum post about it that got a lot of flack, but maybe they were onto something.

Here it is: https://community.openai.com/t/did-chatgpt-4o-get-progressiv...

[ Parent | on HN ]

Re:OpenAI's new "Orion" model reportedly shows small gains o... by Der_Einzige on Monday November 11, 2024 @08:10AM (#42106857)

I’m going to laugh so hard if we finally find out that quantization really does hurt models, and all the claims of 99.9999% at 4bit or lower precision really were full of it!

Of all the gaslighting I see in Ml/AI, the idea that quantization is almost free is up there. I don’t buy it for one second at all, no matter how many charts you try to show me implying that the logprobs are 99% the same. Sure, maybe until you go to 10K tokens context window!

[ Parent | on HN ]

Re:OpenAI's new "Orion" model reportedly shows small gains o... by Jensson on Monday November 11, 2024 @10:51AM (#42108000)

Its probably kind of like dementia, 99% of what they say is like before they had early dementia but its still a big problem.

[ Parent | on HN ]