News for hackers. Stuff that matters.
Re:OpenAI dropped the price of o3 by 80% by behnamoh on Tuesday June 10, 2025 @02:07PM (#44239601)
how do we know it's not a quantized version of o3? what's stopping these firms from announcing the full model to perform well on the benchmarks and then gradually quantizing it (first at Q8 so no one notices, then Q6, then Q4, ...).

I have a suspicion that's how they were able to get gpt-4-turbo so fast. In practice, I found it inferior to the original GPT-4 but the company probably benchmaxxed the hell out of the turbo and 4o versions so even though they were worse models, users found them more pleasing.

Re:OpenAI dropped the price of o3 by 80% by lispisok on Tuesday June 10, 2025 @02:15PM (#44239674)
I swear every time a new model is released it's great at first but then performance gets worse over time. I figured they were fine-tuning it to get rid of bad output which also nerfed the really good output. Now I'm wondering if they were quantizing it.
Re:OpenAI dropped the price of o3 by 80% by Tiberium on Tuesday June 10, 2025 @02:40PM (#44239928)
I've heard lots of people say that, but no objective reproducible benchmarks confirm such a thing happening often. Could this simply be a case of novelty/excitement for a new model fading away as you learn more about its shortcomings?
Re:OpenAI dropped the price of o3 by 80% by Kranar on Tuesday June 10, 2025 @03:10PM (#44240288)
I used to think the models got worse over time as well but then I checked my chat history and what I noticed isn't that ChatGPT gets worse, it's that my standards and expectations increase over time.

When a new model comes out I test the waters a bit with some more ambitious queries and get impressed when it can handle them reasonably well. Over time I take it for granted and then just expect it to be able to handle ever more complex queries and get dissappointed when I hit a new limit.

Re:OpenAI dropped the price of o3 by 80% by echelon on Tuesday June 10, 2025 @03:32PM (#44240519)
Re-run your historical queries, or queries that are similarly shaped.
Re:OpenAI dropped the price of o3 by 80% by throwaway314155 on Tuesday June 10, 2025 @05:28PM (#44241648)
Sounds like a _whole_ thing.
Re:OpenAI dropped the price of o3 by 80% by sakesun on Tuesday June 10, 2025 @06:05PM (#44241977)
They could cache that :)
Re:OpenAI dropped the price of o3 by 80% by echelon on Wednesday June 11, 2025 @10:13AM (#44247853)
That would make for a very interesting timing attack.
Re:OpenAI dropped the price of o3 by 80% by herval on Tuesday June 10, 2025 @02:46PM (#44240006)
there's definitely measurements (eg https://hdsr.mitpress.mit.edu/pub/y95zitmz/release/2 ) but I imagine they're rare because those benchmarks are expensive, so nobody keeps running them all the time?

Anecdotally, it's quite clear that some models are throttled during the day (eg Claude sometimes falls back to "concise mode" - with and without a warning on the app).

You can tell if you're using Windsurf/Cursor too - there are times of the day where the models constantly fail to do tool calling, and other times they "just work" (for the same query).

Finally, there's cases where it was confirmed by the company, like Gpt-4o's sycopanth tirade that very clearly impacted its output (https://openai.com/index/sycophancy-in-gpt-4o/)

Re:OpenAI dropped the price of o3 by 80% by glitch253 on Tuesday June 10, 2025 @04:08PM (#44240928)
Cursor / Windsurf's degraded functionality is exactly why I created my own system:

https://github.com/mpfaffenberger/code_puppy

Re:OpenAI dropped the price of o3 by 80% by Deathmax on Tuesday June 10, 2025 @03:22PM (#44240417)
Your linked article is specifically comparing two different versioned snapshots of a model and not comparing the same model across time.

You've also made the mistake of conflating what's served via API platforms which are meant to be stable, and frontends which have no stability guarantees, and are very much iterated on in terms of the underlying model and system prompts. The GPT-4o sycophancy debacle was only on the specific model that's served via the ChatGPT frontend and never impacted the stable snapshots on the API.

I have never seen any sort of compelling evidence that any of the large labs tinkers with their stable, versioned model releases that are served via their API platforms.

Re:OpenAI dropped the price of o3 by 80% by herval on Tuesday June 10, 2025 @03:32PM (#44240518)
Please read it again. The article is clearly comparing gpt4 to gpt4, and gpt3.5 to gpt3.5, in march vs june 2023
Re:OpenAI dropped the price of o3 by 80% by Deathmax on Tuesday June 10, 2025 @03:38PM (#44240587)
I did read it, and I even went to their eval repo.

> At the time of writing, there are two major versions available for GPT-4 and GPT-3.5 through OpenAI’s API, one snapshotted in March 2023 and another in June 2023.

openaichat/gpt-3.5-turbo-0301 vs openaichat/gpt-3.5-turbo-0613, openaichat/gpt-4-0314 vs openaichat/gpt-4-0613. Two _distinct_ versions of the model, and not the _same_ model over time like how people like to complain that a model gets "nerfed" over time.

Re:OpenAI dropped the price of o3 by 80% by drewnick on Tuesday June 10, 2025 @03:18PM (#44240365)
I feel this too. I swear some of the coding Claude Code does on weekends is superior to the weekdays. It just has these eureka moments every now and then.
Re:OpenAI dropped the price of o3 by 80% by herval on Tuesday June 10, 2025 @03:20PM (#44240388)
Claude has been particularly bad since they released 4.0. The push to remove 3.7 from Windsurf hasn’t helped either. Pretty evident they’re trying to force people to pay for Claude Code…

Trusting these LLM providers today is as risky as trusting Facebook as a platform, when they were pushing their “opensocial” stuff

Re:OpenAI dropped the price of o3 by 80% by bobxmax on Tuesday June 10, 2025 @03:44PM (#44240657)
My suspicion is it's the personalization. Most people have things like 'memory' on, and as the models increasingly personalize towards you, that personalization is hurting quality rather than helping it.

Which is why the base model wouldn't necessarily show differences when you benchmarked them.

Re:OpenAI dropped the price of o3 by 80% by 85392_school on Tuesday June 10, 2025 @02:45PM (#44239997)
I think it's an illusion. People have been claiming it since the GPT-4 days, but nobody's ever posted any good evidence to the "model-changes" channel in Anthropic's Discord. It's probably just nostalgia.
Re:OpenAI dropped the price of o3 by 80% by cainxinth on Tuesday June 10, 2025 @04:31PM (#44241144)
I assumed it was because the first week revealed a ton of safety issues that they then "patched" by adjusting the system prompt, and thus using up more inference tokens on things other than the user's request.
Re:OpenAI dropped the price of o3 by 80% by colordrops on Tuesday June 10, 2025 @07:37PM (#44242693)
It's probably less often quantizing and more often adding more and more to their hidden system prompt to address various issues and "issues", and as we all know, adding more context sometimes has a negative effect.
Re:OpenAI dropped the price of o3 by 80% by tshaddox on Tuesday June 10, 2025 @09:15PM (#44243280)
Yeah, it’s almost certainly hallucination (by the human user).
Re:OpenAI dropped the price of o3 by 80% by nabla9 on Tuesday June 10, 2025 @02:26PM (#44239775)
It seems that least Google is overselling their compute capacity.

You pay monthly fee, but Gemini is completely jammed 5-6 hours when North America is working.

Re:OpenAI dropped the price of o3 by 80% by baq on Tuesday June 10, 2025 @02:44PM (#44239985)
Gemini is simply that good. I’m trying out Claude 4 every now and then and go back to Gemini to fix its mess…
Re:OpenAI dropped the price of o3 by 80% by energy123 on Tuesday June 10, 2025 @03:19PM (#44240387)
Gemini is the best model in the world. Gemini is the worst web app in the world. Somehow those two things are coexisting. The web devs in their UI team have really betrayed the hard work of their ML and hardware colleagues. I don't say this lightly - I say this after having paid attention to critical bugs, more than I can count on one hand, that persisted for over a year. They either don't care or are grossly incompetent.
Re:OpenAI dropped the price of o3 by 80% by thorum on Tuesday June 10, 2025 @03:41PM (#44240611)
Try AI Studio if you haven’t already: https://aistudio.google.com/
Re:OpenAI dropped the price of o3 by 80% by koakuma-chan on Tuesday June 10, 2025 @04:14PM (#44240990)
Re:OpenAI dropped the price of o3 by 80% by nabla9 on Tuesday June 10, 2025 @04:25PM (#44241094)
Well said.

Google is best in pure AI research, both quality and volume. They have sucked at productization for years. Not not just AI but other products as well. Real mystery.

Re:OpenAI dropped the price of o3 by 80% by energy123 on Tuesday June 10, 2025 @04:36PM (#44241191)
I don't understand why they can't just make it fast and go through the bug reports from a year ago and fix them. Is it that hard to build a box for users to type text into without it lagging for 5 seconds or throwing a bunch of errors?
Re:OpenAI dropped the price of o3 by 80% by baq on Thursday June 12, 2025 @02:35AM (#44254805)
If it doesn’t make sense, it makes sense. Nobody will get their promo by ‘fixing bugs’.
Re:OpenAI dropped the price of o3 by 80% by fasterthanlime on Tuesday June 10, 2025 @02:54PM (#44240098)
Funny, I have the exact opposite experience! I use Claude to fix Gemini’s mess.
Re:OpenAI dropped the price of o3 by 80% by symfoniq on Tuesday June 10, 2025 @03:00PM (#44240156)
Maybe LLMs just make messes.
Re:OpenAI dropped the price of o3 by 80% by hgomersall on Tuesday June 10, 2025 @02:58PM (#44240134)
I heard that, but I'm getting consistent garbage from Gemini.
Re:OpenAI dropped the price of o3 by 80% by dayjah on Tuesday June 10, 2025 @03:18PM (#44240379)
For code? Use the context7 mcp.
Re:OpenAI dropped the price of o3 by 80% by edzitron on Tuesday June 10, 2025 @04:32PM (#44241152)
When you say "jammed," how do you mean?
Re:OpenAI dropped the price of o3 by 80% by JamesBarney on Tuesday June 10, 2025 @02:53PM (#44240087)
I'm pretty sure this is just a psychological phenomenon. When a new model is released all the capabilities the new model has that the old model lacks are very salient. This makes it seem amazing. Then you get used to the model, push it to the frontier, and suddenly the most salient memories of the new model are it's failures.

There are tons of benchmarks that don't show any regressions. Even small and unpublished ones rarely show regressions.

Re:OpenAI dropped the price of o3 by 80% by mhitza on Tuesday June 10, 2025 @02:32PM (#44239843)
That was my suspicion when I first deleted my account, when it felt the output got worse in ChatGPT and I found highly suspicious when I saw an errand davinci model keyword in the chatgpt url.

Now I'm feeling similarly with their image generation (which is the only reason I created a paid account two months ago, and the output looks more generic by default).

Re:OpenAI dropped the price of o3 by 80% by beering on Tuesday June 10, 2025 @09:01PM (#44243206)
Are you able to quantify how quickly your perception gets skewed by how long you use the models?
Re:OpenAI dropped the price of o3 by 80% by mhitza on Wednesday June 11, 2025 @01:23PM (#44249782)
I can't quantity it for my past experience, that was more than a year ago, and I wasn't using ChatGPT daily at the time either.

This time around it felt pretty stark. I used ChatGPT to create at most 20 different image compositions. And after a couple of good ones at first, it felt worse after. One thing I've noticed recently is that when working on vector art compositions, the results start more simplistic, and often enough look like clipart thrown together. This wasn't my experience first time around. Might be temperature tweaks, or changes in their prompt that lead to this effect. Might be some random seed data they use, who knows.

Re:OpenAI dropped the price of o3 by 80% by codr7 on Tuesday June 10, 2025 @02:21PM (#44239726)
If they could, they would.

All the proof I need.

Re:OpenAI dropped the price of o3 by 80% by daseiner1 on Tuesday June 10, 2025 @02:24PM (#44239753)
It's still a very competitive marketplace
Re:OpenAI dropped the price of o3 by 80% by mathgradthrow on Tuesday June 10, 2025 @02:40PM (#44239940)
honestly refreshing take.
Re:OpenAI dropped the price of o3 by 80% by bboygravity on Tuesday June 10, 2025 @02:34PM (#44239871)
But OpenAI breathes honesty. They're open source! They would never do such a thing. /s
Re:OpenAI dropped the price of o3 by 80% by solfox on Tuesday June 10, 2025 @02:32PM (#44239838)
I have seen this behavior as well.
Re:OpenAI dropped the price of o3 by 80% by JoshuaDavid on Tuesday June 10, 2025 @08:33PM (#44243047)
I suspect what's happening is that lots of people have a collection of questions / private evals that they've been testing on every new model, and when a new model comes out it sometimes can answer a question that previous models couldn't. So that selects for questions where the new model is at the edge of its capabilities and probably got lucky. But when you come up with a new question, it's generally going to be on the level of the questions the new model is newly able to solve.

Like I suspect if there was a "new" model which was best-of-256 sampling of gpt-3.5-turbo that too would seem like a really exciting model for the first little bit after it came out, because it could probably solve a lot of problems current top models struggle with (which people would notice immediately) while failing to do lots of things that are a breeze for top models (which would take people a little bit to notice).

Re:OpenAI dropped the price of o3 by 80% by beering on Tuesday June 10, 2025 @09:00PM (#44243202)
It’s easy to measure the models getting worse, so you should be suspicious that nobody who claims this has scientific evidence to back it up.
Re:OpenAI dropped the price of o3 by 80% by hyperknot on Tuesday June 10, 2025 @03:37PM (#44240574)
I got 700+ tokens/sec on o3 after the announcement, I suspect it's very much a quantized version.

https://x.com/hyperknot/status/1932476190608036243

Re:OpenAI dropped the price of o3 by 80% by dist-epoch on Tuesday June 10, 2025 @04:07PM (#44240903)
Or maybe they just brought online much faster much cheaper hardware.
Re:OpenAI dropped the price of o3 by 80% by az226 on Tuesday June 10, 2025 @08:52PM (#44243160)
Or they are using a speedy add-on decoder.
Re:OpenAI dropped the price of o3 by 80% by zackangelo on Tuesday June 10, 2025 @04:27PM (#44241113)
Is that input tokens or output tokens/s?
Re:OpenAI dropped the price of o3 by 80% by beering on Tuesday June 10, 2025 @09:02PM (#44243212)
Do you also have numbers on intelligence before and after?
Re:OpenAI dropped the price of o3 by 80% by CSMastermind on Tuesday June 10, 2025 @02:37PM (#44239898)
This is almost certainly what they're doing and rebranding the original o3 model as "o3-pro"
Re:OpenAI dropped the price of o3 by 80% by tedsanders on Tuesday June 10, 2025 @04:16PM (#44241005)
Nope, not what we’re doing.

o3 is still o3 (no nerfing) and o3-pro is new and better than o3.

If we were lying about this, it would be really easy to catch us - just run evals.

(I work at OpenAI.)

Re:OpenAI dropped the price of o3 by 80% by bn-l on Tuesday June 10, 2025 @05:41PM (#44241768)
Not quantized?
Re:OpenAI dropped the price of o3 by 80% by tedsanders on Tuesday June 10, 2025 @06:29PM (#44242198)
Not quantized. Weights are the same.

If we did change the model, we'd release it as a new model with a new name in the API (e.g., o3-turbo-2025-06-10). It would be very annoying to API customers if we ever silently changed models, so we never do this [1].

[1] `chatgpt-4o-latest` being an explicit exception

Re:OpenAI dropped the price of o3 by 80% by thegeomaster on Tuesday June 10, 2025 @07:27PM (#44242629)
Google could at least learn something from this attitude, given their recent 03-25 -> 05-06 model alias switcharoo with 0 notice :)
Re:OpenAI dropped the price of o3 by 80% by johnb231 on Tuesday June 10, 2025 @09:09PM (#44243244)
That is a preview / beta model with no expectation of stability. Google did nothing wrong there. No one should be using a preview model in production.
Re:OpenAI dropped the price of o3 by 80% by thegeomaster on Tuesday June 10, 2025 @09:54PM (#44243471)
Hard disagree. Of course technically they didn't do anything explicitly against the public guidance (the checks and balances would never let them), but naming a model with a date very strongly implies immutability.

It's the same logic of why UB in C/C++ isn't a license to do whatever the compiler wants. We're humans and we operate on implications, common-sense assumptions and trust.

Re:OpenAI dropped the price of o3 by 80% by johnb231 on Tuesday June 10, 2025 @11:18PM (#44243892)
The model is labelled as Preview. There are no guarantees of stability or availability for Preview models. Not intended for production workloads.

https://cloud.google.com/products?hl=en#product-launch-stage...

"At Preview, products or features are ready for testing by customers. Preview offerings are often publicly announced, but are not necessarily feature-complete, and no SLAs or technical support commitments are provided for these. Unless stated otherwise by Google, Preview offerings are intended for use in test environments only. The average Preview stage lasts about six months."

Re:OpenAI dropped the price of o3 by 80% by 0xbadcafebee on Tuesday June 10, 2025 @11:54PM (#44244049)
There's a very large gulf between "what makes sense to Google" and "what makes sense to Human Beings". I have so many rants about Google's poor treatment of "customers" that they feel like Oracle to me now. Like every time I use them, I'm really just falling prey to my own misguided idea that this time I won't get screwed over.
Re:OpenAI dropped the price of o3 by 80% by johnb231 on Wednesday June 11, 2025 @12:18AM (#44244159)
The users aren't random "human beings" in this case. They are professional software developers who are expected to understand the basics. Deploying that model into production shows a lack of basic competence. It is clearly marked "preview" and is for test only.
Re:OpenAI dropped the price of o3 by 80% by 0xbadcafebee on Wednesday June 11, 2025 @08:36AM (#44246966)
That may be true, but it doesn't make the customer's claims not true. What Google did was counter-intuitive. That's a fact. Pointing at some fine print and saying "uhh actually, technically it's your stupid human brain is the problem, not us! we technically are allowed to do anything we want, just look at the fine print!!" does not make things better. We are human beings; we are flawed. That much should be obvious to any human organization. If you don't know how to make things that don't piss off human beings, the problem isn't with the humans.

If the "preview release" you were using was v0.3, and suddenly it started being v0.6 without warning, that would be insane. The only point of providing a version number is to give people an indicator of consistency. The datestamp is a version number. If they didn't want us to expect consistency, they should not have given it a version number. That's the whole point of rolling release branches, they have no version. You don't have "v2.0" of a rolling release, you just have "latest". They fucked up by giving it a datestamp.

This is an extremely old and well-known problem with software interfaces. Either you version it or you don't. If you do version it, and change it, you change the version, and give people dependent on the old version some time to upgrade. Otherwise it breaks things, and that pisses people off. The alternative is not versioning it, which is a signal that there is no consistency to be expected. Any decent software developer should have known all this.

And while I'm at it: what's with the name flip-flopping? In 2014, GCP issued a PR release explaining It was no longer using "Preview", but "Alpha" and "Beta" (https://cloudplatform.googleblog.com/2014/10/new-release-pha...). But the link you showed earlier says "Alpha" and "Beta" are now deprecated. But no PR release? I guess that's our bad for not constantly reading the fine print and expecting it to revert back to something from 11 years ago.

Re:OpenAI dropped the price of o3 by 80% by refulgentis on Tuesday June 10, 2025 @11:27PM (#44243940)
There hasn't been a non-preview Gemini since...November? The previews are the same as everyone else's release cadance, "preview" is just a magic wand that meant the Launchcal (google's internal signoff tool, i.e. "wave will never happen again) needs less signoffs. Then it got to the point date-pinned models were getting swapped in, in the name of doing us a favor, and it's a...novel idea, we can both agree at the least.

I bet someone at Google would be a bit surprised to see someone jumping to legalese to act like this...novelty...is inherently due to the preview status, and based on anything more than a sense that there's no net harm done to us if it costs the same and is better.

I'm not sure they're wrong.

But it also leads to a sort of "nobody knows how anything works because we have 2^N configs and 5 bits" - for instance, 05-06 was also upgraded to 06-05. Except it wasn't, if you sent variable thinking to 05-06 after upgrade it'd fail. (and don't get me started on the 5 different thinking configurations for Gemini 2.5 flash thinking vs. gemini 05-06 vs. 06-05 and 0 thinking)

Re:OpenAI dropped the price of o3 by 80% by johnb231 on Tuesday June 10, 2025 @11:49PM (#44244025)
I honestly have no idea what you are trying to say.

It's a preview model - for testing only, not for production. Really not that complicated.

Re:OpenAI dropped the price of o3 by 80% by refulgentis on Wednesday June 11, 2025 @12:10AM (#44244126)
So you don't have anything to contribute beyond, and aren't interested in anything beyond, citing of terms?

Why are you in the comments section of a engineering news site?

(note: beyond your, excuse me while I'm direct now, boorish know-nothing reply, the terms you are citing have nothing to do with the thing people are actually discussing around you, despite your best efforts. It doesn't say "we might swap in a new service, congrats!", nor does it have anything to say about that. Your legalese at most describes why they'd pull 05-06, not forward 05-06 to 06-05. This is a novel idea.)

Re:OpenAI dropped the price of o3 by 80% by johnb231 on Wednesday June 11, 2025 @12:13AM (#44244145)
This case was simply a matter of people not understanding the terms of service. There is nothing more to be said. It's that simple. The "engineers" should know that before deploying to prod. Basic competence.

And I mean I genuinely do not understand what you are trying to say. Couldn't parse it.

Re:OpenAI dropped the price of o3 by 80% by refulgentis on Wednesday June 11, 2025 @12:16AM (#44244153)
John, do you understand that the thing you're quoting says "We reserve the right to pull things", not "We reserve the right to swap in a new service"? Do you understand that even if it was, it was something else entirely?

I have exactly your attitude about their cavalier use of preview for all things Gemini, but I've also been on this site for 15 years and am a bit wow'd by your interlocution style here -- it's quite rare to see someone flip "the 3P provider swapped the service on us!" into "well they said they could turn it off, of course you should expect it to be swapped for the first time ever! insert lame sneer about enfineers

Re:OpenAI dropped the price of o3 by 80% by johnb231 on Wednesday June 11, 2025 @12:21AM (#44244175)
How is this so hard to understand? It's a preview service for testing only, not intended for production.

I am done with this thread. We are going around in circles.

Re:OpenAI dropped the price of o3 by 80% by refulgentis on Wednesday June 11, 2025 @12:23AM (#44244191)
Well, no. Well, sure. You're done, but we're not going in circles. It'd just do too much damage to you to have to answer the simple question "Where does the legalese say they can swap in a new service?", so you have to pretend this is circular and just all-so-confusing, de facto, we have to pretend it is confusing and/or obviously wrong to use any Gemini 2+ at all.

It's a cute argument, as I noted, I'm emotionally sympathetic to it even, it's my favorite "get off my lawn." However, I've also been on the Internet long enough to know you write back, at length, when people try anti-intellectualism and why-are-we-even-talking-about-this as interaction.

Re:OpenAI dropped the price of o3 by 80% by johnb231 on Wednesday June 11, 2025 @12:35AM (#44244232)
https://cloud.google.com/terms/service-terms

"b. Disclaimer. PRE-GA OFFERINGS ARE PROVIDED “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES OR REPRESENTATIONS OF ANY KIND. Pre-GA Offerings (i) may be changed, suspended or discontinued at any time without prior notice to Customer and (ii) are not covered by any SLA or Google indemnity. Except as otherwise expressly indicated in a written notice or Google documentation, (A) Pre-GA Offerings are not covered by TSS, and (B) the Data Location Section above will not apply to Pre-GA Offerings."

Re:OpenAI dropped the price of o3 by 80% by anoncareer0212 on Wednesday June 11, 2025 @12:37AM (#44244244)
John, you found new legal-ese, but this isn't the legalese you shared that supposedly said they'd swap in a new service.

Do you have any comment on where in the original quote it said that?

If not, do you have any comment on if your understanding of the situation may have, well, "evolved" over the course of the thread?

John, is it okay to complain about something even if its contractually allowed?

John, is anyone here saying "people should rely on every property of preview models"?

[dead]
Re:OpenAI dropped the price of o3 by 80% by anoncareer0212 on Wednesday June 11, 2025 @12:40AM (#44244253)
Been here for 15 years, and there's standards for interaction, especially for 19 day old accounts. I recommend other sites if you're expecting to be dismissive and rude without strong intellectual pushback.
Re:OpenAI dropped the price of o3 by 80% by Gigablah on Wednesday June 11, 2025 @02:08AM (#44244645)
This is just “f—- off” dressed in formal language, so not much better.
Re:OpenAI dropped the price of o3 by 80% by johnb231 on Wednesday June 11, 2025 @12:43AM (#44244271)
I'm not the one being rude here. That person is psychotic.

This site is filled with mentally ill people.

Re:OpenAI dropped the price of o3 by 80% by lcnPylGDnU4H9OF on Wednesday June 11, 2025 @08:46AM (#44247051)
> And I mean I genuinely do not understand what you are trying to say. Couldn't parse it.

It’s always worth considering that this may be your problem. If you still don’t get it, the only valuable reply is one which asks a question. Also, including “it’s not that complicated” only serves to inflame.

Re:OpenAI dropped the price of o3 by 80% by ant6n on Tuesday June 10, 2025 @07:25PM (#44242608)
It was definitely annoying when o1 disappeared over night, my impression is that was better at some tasks than o3.
Re:OpenAI dropped the price of o3 by 80% by linsomniac on Tuesday June 10, 2025 @08:47PM (#44243128)
>we'd release it as a new model with a new name

Speaking of a new name. I'll donate the API credits to run a "choose a naming scheme for AI models that isn't confusing AF" for OpenAI.

Re:OpenAI dropped the price of o3 by 80% by MattDaEskimo on Tuesday June 10, 2025 @06:53PM (#44242410)
What's with the dropped benchmark performance compared to the original o3 release? It was disappointing to not see o4-mini on it as well
Re:OpenAI dropped the price of o3 by 80% by refulgentis on Tuesday June 10, 2025 @08:01PM (#44242855)
What dropped benchmark performance?
Re:OpenAI dropped the price of o3 by 80% by MattDaEskimo on Wednesday June 11, 2025 @10:06AM (#44247785)
o3 scores noticeably worse on benchmarks compared to its original announcement benchmarks
Re:OpenAI dropped the price of o3 by 80% by refulgentis on Wednesday June 11, 2025 @12:07PM (#44249058)
Any link / source / anything? You got quite an opportunity here, OpenAI employee claiming there's no difference and you got something that shows there is.
Re:OpenAI dropped the price of o3 by 80% by MattDaEskimo on Wednesday June 11, 2025 @01:22PM (#44249773)
Yes, the original announcement for o3 and o4-mini:

https://openai.com/index/introducing-o3-and-o4-mini/

o3 scored 91.6 on AIME 2024. 83.3 on GPQA

o4-mini scored 93.4, 81.4 GPQA

Then, the new announcement

https://help.openai.com/en/articles/6825453-chatgpt-release-...

o3 scored 90 on AIME 2024, 81 GPQA

o4-mini wasn't measured

---

Codeforces is the same, but they have a footnote that they're using a different dataset due to saturation, but still have no grounding model to compare with

Re:OpenAI dropped the price of o3 by 80% by tedsanders on Sunday June 15, 2025 @10:00PM (#44286129)
First post measures o3 at high reasoning effort. Second measures at medium reasoning effort. It’s the same model, then and now.
Re:OpenAI dropped the price of o3 by 80% by energy123 on Tuesday June 10, 2025 @11:32PM (#44243957)
Is it o3 (low), o3 (medium) or o3 (high)? Different model names have crept into the various benchmarks over the last few months.
Re:OpenAI dropped the price of o3 by 80% by tedsanders on Wednesday June 11, 2025 @12:55AM (#44244324)
o3 is a model, and reasoning effort (high/medium/low) is a parameter that goes into the model.

o3 pro is a different thing - it’s not just o3 with maximum remaining effort.

Re:OpenAI dropped the price of o3 by 80% by fragmede on Wednesday June 11, 2025 @04:48AM (#44245538)
Could someone there maybe possibly use, oh I dunno, ChatGPT and come up with some better product names?
Re:OpenAI dropped the price of o3 by 80% by tauntz on Wednesday June 11, 2025 @05:16AM (#44245678)
Why's it called o3 then if it's a different thing? There's already a rather extreme amount of confusion with the model names and it's not clear _at all_ which model would be "the best" in terms of response quality.

Here's the current state with version numbers as far as I can piece it together (using my best guess at naming of each component of the version identifier. Might be totally wrong tho):

1) prefix (optional): "gpt-", "chatgpt-"

2) family (required): o1, o3, o4, 4o, 3.5, 4, 4.1, 4.5,

3) quality? (optional): "nano", "mini", "pro", "turbo"

4) type (optional): "audio", "search"

5) lifecycle (optional): "preview", "latest"

6) date (optional): 2025-04-14, 2024-05-13, 1106, 0613, 0125, etc (I assume the last ones are a date without a year for 2024?)

7) size (optional): "16k"

Some final combinations of these version number components are as small as 1 ("o3") or as large as 6 ("gpt-4o-mini-search-preview-2024-12-17").

Given this mess, I can't blame people assuming that the "best" model is the one with the "biggest" number, which would rank the model families as: 4.5 (best) > 4.1 > 4 > 4o > o4 > 3.5 > o3 > o1 (worst).

Re:OpenAI dropped the price of o3 by 80% by rat9988 on Wednesday June 11, 2025 @08:20AM (#44246844)
o3 and o3-pro aren't the same thing still makes sense though.
Re:OpenAI dropped the price of o3 by 80% by energy123 on Wednesday June 11, 2025 @09:28AM (#44247424)
My guess is this comes from an org structure where you have multiple "pods" working on different research. Who comes up with the next shippable model and when that happens is kind of random and the chaotic naming system comes from that. It's just my speculation and could be wildly wrong.
Re:OpenAI dropped the price of o3 by 80% by tedsanders on Wednesday June 11, 2025 @11:38AM (#44248790)
o3 pro is based on o3 and its style and outputs will be quite similar to o3.

As an analogy, think of it like this:

o3-low ~ Ford Mustang with the accelerator gently pressed

o3-medium ~ Ford Mustang with the accelerator pressed

o3-high ~ Ford Mustang with the accelerator heavily pressed

o3 pro ~ Ford Mustang GT

Even though a Mustang GT is a different car than a Mustang, you don’t give it a totally different name (eg Palomino). The similarity in name signals it has a lot of the same characteristics but a souped up engine. Same for o3 pro.

Fun fact: before GPT-4, we had a unified naming scheme for models that went {modality}-{size}-{version}, which resulted in names like text-davinci-002. We considered launching GPT-4 as something like text-earhart-001, but since everyone was calling it GPT-4 anyway, we abandoned that system to use the name GPT-4 that everyone had already latched onto. Kind of funny how our original unified naming scheme made room for 999 versions, but we didn't make it past 3.

Re:OpenAI dropped the price of o3 by 80% by stonogo on Wednesday June 11, 2025 @12:09PM (#44249072)
This analogy might work better if the Mustang GT weren't, in fact, the same car as the Mustang. It's just a trim level, not a different car.
Re:OpenAI dropped the price of o3 by 80% by dwohnitmok on Wednesday June 11, 2025 @01:23PM (#44249780)
Can you elaborate on what you mean that o3 pro is a GT? In particular I don't understand how to reconcile what you're saying that o3 pro is in some way fundamentally different from o3 (albeit based on o3) with this tweet:

> As o3-pro uses the same underlying model as o3, full safety details can be found in the o3 system card.

https://x.com/OpenAI/status/1932530423911096508

Re:OpenAI dropped the price of o3 by 80% by tedsanders on Wednesday June 11, 2025 @01:40PM (#44249947)
Yeah, I totally get the confusion here. Unfortunately I can't give the recipe behind our models, so there's going to be some irreducible blurriness here, but the following statements are all true:

- o3 pro is based on o3

- o3 pro uses the same underlying model as o3

- o3 pro is similar to o3, but is a distinct thing that's smarter and slower

- o3 pro is not o3 with longer reasoning

In my analogy, o3 pro vs o3 is more than just an input parameter (e.g., not just the accelerator input) but less than a full difference in model (e.g., Ford Mustang vs F150). It's in between, kind of like car trim with the same body but a stronger engine. Imperfect analogy, and I apologize if this doesn't feel like it adds any clarity. At the end of the day, it doesn't really matter how it works - what matters is if people find it worth using.

Re:OpenAI dropped the price of o3 by 80% by csomar on Wednesday June 11, 2025 @12:53AM (#44244316)
I think the parent-parent poster has explained why we can't trust you (and work on OpenAI doesn't help they way you think it does).

I didn't read the ToS, like everyone else, but my guess is that degrading model performance at peak times will be one of the things that can slip through. We are not suggesting you are running a different model but that you are quantizing it so that you can support more people.

This can't happen with Open weight models where you put the model, allocate the memory and run the thing. With OpenAI/Claude, we don't know the model running, how large it is, what it is running on, etc... None of that is provided and there is only one reason that I can think of: to be able to reduce resources unnoticed.

Re:OpenAI dropped the price of o3 by 80% by rfoo on Wednesday June 11, 2025 @05:32AM (#44245768)
An (arbitrarily) quantized model is a totally different model, compared to the original.
Re:OpenAI dropped the price of o3 by 80% by Reubachi on Wednesday June 11, 2025 @09:49AM (#44247605)
I'm not totally sure how you at this point in your online presence associate someone stating their job as a "brag" and not what it really is, providing transparency/disclosure before stating their thoughts.

This is HN and not reddit.

"I didn't read the ToS, like everyone else, but my guess..."

Ah, there it is.

Re:OpenAI dropped the price of o3 by 80% by fastball on Wednesday June 11, 2025 @02:48AM (#44244846)
Anecdotal, but about a week ago I noticed a sharp drop in o3 performance. For many tasks I will compare Gemini 2.5 Pro with o3, running the same prompt in both. Generally for my personal use o3 and G2.5P have been neck-and neck over the last months, with responses I have been very happy with.

However starting from a week ago, the o3 responses became noticeably worse, with G2.5P staying about the same (in terms of what I've come to expect from the two models).

This alongside the news that you guys have decreased the price of o3 by 80% does really make it feel like you've quantized the model or knee-capped thinking or something. If you say it is wholly unchanged I'll believe you, but not sure how else to explain the (admittedly subjective) performance drop I've experienced.

Re:OpenAI dropped the price of o3 by 80% by IanCal on Wednesday June 11, 2025 @06:49AM (#44246200)
Are you sure you're using the same models? G2.5P updated almost exactly a week ago.
Re:OpenAI dropped the price of o3 by 80% by fastball on Wednesday June 11, 2025 @11:18PM (#44253999)
G2.5P might've updated, but that's not the model I noticed a difference. o3 seemed noticeably dumber in isolation, not just compared to G2.5P.

But yes, perhaps the answer is that about a week ago I started asking subconsciously harder questions, and G2.5P handled them better because it had just been improved, while o3 had not so it seemed worse. Or perhaps G2.5P has always had more capacity than o3, and I wasn't asking hard enough questions to notice a difference before.

Re:OpenAI dropped the price of o3 by 80% by fny on Wednesday June 11, 2025 @07:20AM (#44246398)
Unrelated: Can you all come up with a better naming scheme for your models? I feel like this is a huge UX miss.

o4-mini-high o4-mini o3 o3-pro gpt-4o

Oy.

Re:OpenAI dropped the price of o3 by 80% by meta_ai_x on Wednesday June 11, 2025 @11:31AM (#44248718)
Just because you work at openAI doesn't mean you know everything about openAI especially as strategic as nerfing models to save costs
Re:OpenAI dropped the price of o3 by 80% by behnamoh on Tuesday June 10, 2025 @02:38PM (#44239903)
> rebranding the original o3 model as "o3-pro"

interesting take, I wouldn't be surprised if they did that.

Re:OpenAI dropped the price of o3 by 80% by anticensor on Tuesday June 10, 2025 @02:40PM (#44239938)
-pro models appear to be a best-of-10 sampling of the original full size model
Re:OpenAI dropped the price of o3 by 80% by Szpadel on Tuesday June 10, 2025 @03:03PM (#44240205)
how do you sample it behind the scenes? usually best of X means you generate X outputs and you choose best result.

if you could do this automatically, it would be game changer as you could run top 5 best models in parallel and select best answer every time

but it's not practical because you are the bottleneck as you have to read all 5 solutions and compare them

Re:OpenAI dropped the price of o3 by 80% by joshstrange on Tuesday June 10, 2025 @04:09PM (#44240929)
I think the idea is they use another/same model to judge all the results and only return the best one to the user.
Re:OpenAI dropped the price of o3 by 80% by anticensor on Wednesday June 11, 2025 @06:08AM (#44245952)
I think the idea is they just feed each to the RLHF reward model used to train the model and return the most rewarded answer.
Re:OpenAI dropped the price of o3 by 80% by anticensor on Tuesday June 10, 2025 @05:25PM (#44241618)
> if you could do this automatically, it would be game changer as you could run top 5 best models in parallel and select best answer every time

remember they have access to the RLHF reward model, against which they can evaluate all N outputs and have the most "rewarded" answer picked and sent

Re:OpenAI dropped the price of o3 by 80% by spott on Tuesday June 10, 2025 @09:20PM (#44243297)
I believe it is a majority vote kinda thing, rather than a best single result.
Re:OpenAI dropped the price of o3 by 80% by mliker on Tuesday June 10, 2025 @08:45PM (#44243111)
Where are you getting this information? What basis do you have for making this claim? OpenAI, despite its public drama, is still a massive brand and if this were exposed, would tank the company's reputation. I think making baseless claims like this is dangerous for HN
Re:OpenAI dropped the price of o3 by 80% by beering on Tuesday June 10, 2025 @08:58PM (#44243186)
I think Gell-Mann amnesia happens here too, where you can see how wrong HN comments are on a topic you know deeply, but then forget about that when reading the comments on another topic.
Re:OpenAI dropped the price of o3 by 80% by resters on Tuesday June 10, 2025 @02:58PM (#44240142)
It's probably optimized in some way, but if the optimizations degrade performance, let's hope it is reflected in various benchmarks. One alternative hypothesis is that it's the same model, but in the early days they make it think "harder" and run a meta-process to collect training data for reinforcement learning for use on future models.
Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @03:11PM (#44240298)
It's a bit dated now, but it would be cool if people submitted PRs for this one: https://aider.chat/docs/leaderboards/by-release-date.html
Re:OpenAI dropped the price of o3 by 80% by __mharrison__ on Tuesday June 10, 2025 @04:02PM (#44240838)
Dated? This was updated yesterday https://aider.chat/docs/leaderboards/
Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @06:18PM (#44242107)
My link is to the benchmark results _over time_.

The main leaderboard page that you linked to is updated quite frequently, but it doesn't contain multiple benchmarks for the same exact model.

Re:OpenAI dropped the price of o3 by 80% by benterix on Tuesday June 10, 2025 @02:15PM (#44239673)
> users found them more pleasing.

Some users. For me the drop was so huge it became almost unusable for the things I had used it for.

Re:OpenAI dropped the price of o3 by 80% by behnamoh on Tuesday June 10, 2025 @02:35PM (#44239879)
Same here. One of my apps straight out stopped working because the gpt-4o outputs were noticeably worse than the gpt-4 that I built the app based on.
Re:OpenAI dropped the price of o3 by 80% by ants_everywhere on Tuesday June 10, 2025 @02:55PM (#44240104)
Is this what happened to Gemini 2.5 Pro? It used to be very good, but it's started struggling on basic tasks.

The thing that gets me is it seems to be lying about fetching a web page. It will say things are there that were never on any version of the page and it sometimes takes multiple screenshots of the page to convince it that it's wrong.

Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @03:15PM (#44240339)
The Aider discord community has proposed and disproven the theory that 2.5 Pro became worse, several times, through many benchmark runs.

It had a few bugs here or there when they pushed updates, but it didn't get worse.

Re:OpenAI dropped the price of o3 by 80% by ants_everywhere on Tuesday June 10, 2025 @04:36PM (#44241194)
Gemini is objectively exhibiting new behavior with the same prompts and that behavior is unwelcome. It includes hallucinating information and refusing to believe it's wrong.

My question is not whether this is true (it is) but why it's happening.

I am willing to believe the aider community has found that Gemini has maintained approximately equivalent performance on fixed benchmarks. That's reasonable considering they probably use a/b testing on benchmarks to tell them whether training or architectural changes need to be reverted.

But all versions of aider I've tested, including the most recent one, don't handle Gemini correctly so I'm skeptical that they're the state of the art with respect to bench-marking Gemini.

Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @06:24PM (#44242157)
Gemini 2.5 Pro is the highest ranking model on the aider benchmarks leaderboard.

For benchmarks, either Gemini writes code that adheres to the required edit format, builds successfully, and passes unit tests, or it doesn't.

I primarily use aider + 2.5 pro for planning/spec files, and occasionally have it do file edits directly. Works great, other than stopping it mid-execution once in a while.

Re:OpenAI dropped the price of o3 by 80% by code_biologist on Wednesday June 11, 2025 @04:12AM (#44245317)
My use case is mostly creative writing, if that matters.

2.5 Pro 03-25 was insanely good. I suspect it was also very expensive to run. The 05-06 release was a huge regression in quality. They tested a few different variants and some were less bad then others, but overall it was painful to lose access to such a good model. The just released 06-05 version seems to be uniformly better than 05-06, with far fewer "wow this thing is dumb as a rock" failure modes, but it still is not as strong as the first release.

Entirely anecdotally, 06-05 seems to exactly ride the line of "good enough to be the best, but no better than that" presumably to save costs versus the OG 03-25.

In addition, Google is doing something notably different between what you get on AI Studio versus the Gemini site/app. Maybe a different system prompt. There have been a lot of anecdotal comparisons on /r/bard and I do think the AI Studio version is better.

Re:OpenAI dropped the price of o3 by 80% by carter-0 on Tuesday June 10, 2025 @03:09PM (#44240277)
An OpenAI researcher claims it's the exact same model on X: https://x.com/aidan_mclau/status/1932507602216497608
Re:OpenAI dropped the price of o3 by 80% by esafak on Tuesday June 10, 2025 @02:09PM (#44239622)
Are there any benchmarks that track historical performance?
Re:OpenAI dropped the price of o3 by 80% by behnamoh on Tuesday June 10, 2025 @02:10PM (#44239639)
good question, and I don't know of any, although it's a no brainer that someone should make it.

a proxy to that may be the anecdotal evidence of users who report back in a month that model X has gotten dumber (started with gpt-4 and keeps happening, esp. with Anthro and OpenAI models). I haven't heard such anecdotal stories about Gemini, R1, etc.

Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @03:17PM (#44240357)
Aider has one, but it hasn't been updated in months. People kept claiming models were getting worse, but the results proved that they weren't.
Re:OpenAI dropped the price of o3 by 80% by __mharrison__ on Tuesday June 10, 2025 @04:02PM (#44240843)
Re:OpenAI dropped the price of o3 by 80% by vitaflo on Tuesday June 10, 2025 @04:26PM (#44241098)
That Deepseek price is always hilarious to see in these charts.
Re:OpenAI dropped the price of o3 by 80% by SparkyMcUnicorn on Tuesday June 10, 2025 @06:26PM (#44242178)
That's not the one I'm referring to. See my other comments or your sibling comment.
Re:OpenAI dropped the price of o3 by 80% by esafak on Tuesday June 10, 2025 @03:43PM (#44240642)
Re:OpenAI dropped the price of o3 by 80% by risho on Tuesday June 10, 2025 @02:18PM (#44239695)
Quantization is a massive efficiency gain for near negligible drop in quality. If the tradeoff is quantization for an 80 percent price drop I would take that any day of the week.
Re:OpenAI dropped the price of o3 by 80% by behnamoh on Tuesday June 10, 2025 @02:21PM (#44239725)
> for near negligible drop in quality

Hmm, that's evidently and anecdotally wrong:

https://github.com/ggml-org/llama.cpp/discussions/4110

Re:OpenAI dropped the price of o3 by 80% by spiderice on Tuesday June 10, 2025 @02:24PM (#44239755)
You may be right that the tradeoff is worth it, but it should be advertised as such. You shouldn't think you're paying for full o3, even if they're heavily discounting it.
Re:OpenAI dropped the price of o3 by 80% by code_biologist on Wednesday June 11, 2025 @04:02AM (#44245264)
I would like the option to pay for the unquantized version. For creative or story writing (D&D campaign materials and such) quantization seems to end up in much weaker word selection and phrasing. There are small semantic missteps that break the illusion the LLM understands what it's writing. I find it jarring and deeply immersion breaking. I'd prefer prototype prompts on a cheaper quantized version, but I want to be able to spend 50 cents an API call to get golden output.
Re:OpenAI dropped the price of o3 by 80% by jstummbillig on Tuesday June 10, 2025 @02:58PM (#44240140)
You can just give it a go for very little money (in Windsurf it's 1x right now), and see what it does. There is no room for conspiracy here, because you can simple look at what it does. If you don't like it, so won't others, and then people will not use it. People are obviously very capable of (collectively) forming opinions on models, and then vote with their wallet.
Re:OpenAI dropped the price of o3 by 80% by EnPissant on Tuesday June 10, 2025 @03:22PM (#44240415)
The API lists o3 and o3-2025-04-16 as the same thing with the same price. The date based models are set in stone.
Re:OpenAI dropped the price of o3 by 80% by segmondy on Tuesday June 10, 2025 @03:18PM (#44240380)
you don't, so run your own model.
Re:OpenAI dropped the price of o3 by 80% by Bjorkbat on Tuesday June 10, 2025 @04:48PM (#44241292)
Related, when o3 finally came out ARC-AGI updated their graph because it didn’t perform nearly as well as the version of o3 that “beat” the benchmark.

https://arcprize.org/blog/analyzing-o3-with-arc-agi

Re:OpenAI dropped the price of o3 by 80% by beering on Tuesday June 10, 2025 @09:05PM (#44243223)
The o3-preview test was with very expensive amounts of compute, right? I remember it was north of $10k so makes sense it did better
Re:OpenAI dropped the price of o3 by 80% by Bjorkbat on Wednesday June 11, 2025 @08:06AM (#44246736)
Point remains though, they crushed the benchmark using a specialized model that you’ll probably never have access to, whether personally or through a company.

They inflated expectations and then released to the public a model that underperforms

Re:OpenAI dropped the price of o3 by 80% by throwaway314155 on Wednesday June 11, 2025 @09:42AM (#44247543)
They revealed the price points for running those evaluations. IIRC the "high" level of reasoning cost tens of thousands of dollars if not more. I don't think they really inflated expectations. In fact a lot of what we learned is that ARC-AGI probably isn't a very good AGI evaluation (it claims to not be one, but the name suggests otherwise).
Re:OpenAI dropped the price of o3 by 80% by ctoth on Tuesday June 10, 2025 @05:39PM (#44241746)
From the announcement email:

> Today, we dropped the price of OpenAI o3 by 80%, bringing the cost down to $2 / 1M input tokens and $8 / 1M output tokens.

> We optimized our inference stack that serves o3—this is the same exact model, just cheaper.

Re:OpenAI dropped the price of o3 by 80% by smusamashah on Tuesday June 10, 2025 @05:47PM (#44241837)
Hw about testing same input vs output with same seed on different dates. If its a different model it will return different output.
Re:OpenAI dropped the price of o3 by 80% by zomnoys on Tuesday June 10, 2025 @05:50PM (#44241866)
Isn’t this not true since these models run with a non-zero temperature?
Re:OpenAI dropped the price of o3 by 80% by smusamashah on Tuesday June 10, 2025 @06:22PM (#44242143)
You can set the temperature too.
Re:OpenAI dropped the price of o3 by 80% by luke-stanley on Tuesday June 10, 2025 @06:46PM (#44242353)
I think the API has some special IDs to check for reproducibility of the environment.
Re:OpenAI dropped the price of o3 by 80% by tedsanders on Tuesday June 10, 2025 @06:59PM (#44242447)
It's the same model, no quantization, no gimmicks.

In the API, we never make silent changes to models, as that would be super annoying to API developers [1]. In ChatGPT, it's a little less clear when we update models because we don't want to bombard regular users with version numbers in the UI, but it's still not totally silent/opaque - we document all model updates in the ChatGPT release notes [2].

[1] chatgpt-4o-latest is an exception; we explicitly update this model pointer without warning.

[2] ChatGPT Release Notes document our updates to gpt-4o and other models: https://help.openai.com/en/articles/6825453-chatgpt-release-...

(I work at OpenAI.)

Re:OpenAI dropped the price of o3 by 80% by az226 on Tuesday June 10, 2025 @08:51PM (#44243153)
Even classic GPT-4 from March 2023 was quantized to 4.5 bits.
Re:OpenAI dropped the price of o3 by 80% by rfoo on Wednesday June 11, 2025 @05:23AM (#44245720)
I don't work for OAI so obviously I can't say for them. But we don't do this.

We don't make hobbyist mistakes of randomly YOLO trying various "quantization" methods that only happen after all training and claim it a day, at all. Quantization was done before it went live.