News for hackers. Stuff that matters.
Re:Open models by OpenAI by Rhubarrbb on Tuesday August 05, 2025 @01:37PM (#44801371)
Agreed, these models seem relatively mediocre to Qwen3 / GLM 4.5
Re:Open models by OpenAI by modeless on Tuesday August 05, 2025 @01:44PM (#44801495)
Nah, these are much smaller models than Qwen3 and GLM 4.5 with similar performance. Fewer parameters and fewer bits per parameter. They are much more impressive and will run on garden variety gaming PCs at more than usable speed. I can't wait to try on my 4090 at home.

There's basically no reason to run other open source models now that these are available, at least for non-multimodal tasks.

Re:Open models by OpenAI by thegeomaster on Tuesday August 05, 2025 @04:13PM (#44803654)
[delayed]
Re:Open models by OpenAI by tedivm on Tuesday August 05, 2025 @01:55PM (#44801673)
Qwen3 has multiple variants ranging from larger (230B) than these models to significantly smaller (0.6b), with a huge number of options in between. For each of those models they also release quantized versions (your "fewer bits per parameter).

I'm still withholding judgement until I see benchmarks, but every point you tried to make regarding model size and parameter size is wrong. Qwen has more variety on every level, and performs extremely well. That's before getting into the MoE variants of the models.

Re:Open models by OpenAI by modeless on Tuesday August 05, 2025 @02:04PM (#44801846)
The benchmarks of the OpenAI models are comparable to the largest variants of other open models. The smaller variants of other open models are much worse.
Re:Open models by OpenAI by mrbungie on Tuesday August 05, 2025 @02:13PM (#44801952)
I would wait for neutral benchmarks before making any conclusions.
Re:Open models by OpenAI by bigyabai on Tuesday August 05, 2025 @03:25PM (#44803012)
With all due respect, you need to actually test out Qwen3 2507 or GLM 4.5 before making these sorts of claims. Both of them are comparable to OpenAI's largest models and even bench favorably to Deepseek and Opus: https://cdn-uploads.huggingface.co/production/uploads/62430a...

It's cool to see OpenAI throw their hat in the ring, but you're smoking straight hopium if you think there's "no reason to run other open source models now" in earnest. If OpenAI never released these models, the state-of-the-art would not look significantly different for local LLMs. This is almost a nothingburger if not for the simple novelty of OpenAI releasing an Open AI for once in their life.

Re:Open models by OpenAI by modeless on Tuesday August 05, 2025 @03:46PM (#44803320)
> Both of them are comparable to OpenAI's largest models and even bench favorably to Deepseek and Opus

So are/do the new OpenAI models, except they're much smaller.

Re:Open models by OpenAI by UrineSqueegee on Tuesday August 05, 2025 @04:12PM (#44803633)
I'd really wait for additional neutral benchmarks, I asked the 20b model on low reasoning effort which number is larger 9.9 or 9.11 and it got it wrong.

Qwen-0.6b gets it right.

Re:Open models by OpenAI by bigyabai on Tuesday August 05, 2025 @06:36PM (#44805352)
According to the early benchmarks, it's looking like you're just flat-out wrong: https://blog.brokk.ai/a-first-look-at-gpt-oss-120bs-coding-a...
Re:Open models by OpenAI by nialv7 on Wednesday August 06, 2025 @10:28PM (#44819964)
Looks OpenAI's first mover advantages are still alive and well
Re:Open models by OpenAI by sourcecodeplz on Tuesday August 05, 2025 @03:23PM (#44802979)
From my initial web developer test on https://www.gpt-oss.com/ the 120b is kind of meh. Even qwen3-coder 30b-a3b is better. have to test more.
Re:Open models by OpenAI by moralestapia on Tuesday August 05, 2025 @01:53PM (#44801642)
You can always get your $0 back.
Re:Open models by OpenAI by Imustaskforhelp on Tuesday August 05, 2025 @01:58PM (#44801723)
I have never agreed with a comment so much but we are all addicted to open source models now.
Re:Open models by OpenAI by recursive on Tuesday August 05, 2025 @04:08PM (#44803586)
Not all of us. I've yet to get much use out of any of the models. This may be a personal failing. But still.
Re:Open models by OpenAI by satvikpendem on Tuesday August 05, 2025 @02:04PM (#44801836)
Depends on how much you paid for the hardware to run em on
Re:Open models by OpenAI by cvadict on Tuesday August 05, 2025 @03:53PM (#44803403)
Yes, but they are suuuuper safe. /s

So far I have mixed impressions, but they do indeed seem noticeably weaker than comparably-sized Qwen3 / GLM4.5 models. Part of the reason may be that the oai models do appear to be much more lobotomized than their Chinese counterparts (which are surprisingly uncensored). There's research showing that "aligning" a model makes it dumber.

Re:Open models by OpenAI by xwolfi on Tuesday August 05, 2025 @10:42PM (#44807044)
The censorship here in China is only about public discussions / spaces. You cannot like have a website telling you about the crimes of the party. But downloading some compressed matrix re-spouting the said crimes, nobody gives a damn.

We seem to censor organized large scale complaints and viral mind virii, but we never quite forbid people at home to read some generated knowledge from an obscure hard to use software.