News for hackers. Stuff that matters.
Re:Cerebras Deploys Trillion-Parameter AI Models by keyle on Sunday October 25, 2026 @02:13AM (#50228806)
Cerebras is good on paper but all I can reasonably access is "GPT OSS 120b, Qwen 3.8 27B" which aren't very good for my use.

Openrouter on nitro gets me there. Although the TPS is 5X less still.

I have tried GPT OSS 120b on cerebras and it flies. But it's mostly wrong.

Re:Cerebras Deploys Trillion-Parameter AI Models by 2001zhaozhao on Sunday October 25, 2026 @03:56AM (#50229280)
I was a subscriber of their now-unavailable coding plan. The 1000 TPS was very unique at the time, but nowadays their qwen 3.8 gets roflstomped by Mercury 3 at both cost and performance at similar TPS speed. There's not much reason to use them unless you're an enterprise with a custom deal.
Re:Cerebras Deploys Trillion-Parameter AI Models by ekidd on Sunday October 25, 2026 @07:27AM (#50230478)
Qwen3.8 27B is a dense model with expensive K/V caching. It's fantastic if you have 32-48GB of VRAM and want to run 1 or 2 agent sessions locally. But if serving batch inference for many users, it's expensive.

DeepSeek V4 Flash 0731 is dirt cheap at scale, less than 10% of the cost of Qwen3.8 27B. This is because it's an MoE and it uses much less VRAM for KV cache. And it's generally a better model. V4.1 is also cheap at scale, though I've heard mixed reports on the optimized K/V caching.

You should not be running Qwen3.8 27B on an API normally. Maybe Cerberas's hardware is bad at batching or supporting more efficient K/V caching schemes, or something like that?

Re:Cerebras Deploys Trillion-Parameter AI Models by cyanydeez on Sunday October 25, 2026 @09:54AM (#50231632)
If you have 48GB of VRAM, you should be able to get Qwen3.8-Flash-Next running and from my use, it's basically the tipping point. 27B is great for a bit of work, but I have QFN running non stop (when I can gather the correct ideas for it to implement).
Re:Cerebras Deploys Trillion-Parameter AI Models by xmddmx on Sunday October 25, 2026 @10:45AM (#50232102)
What quant and engine are you using? The smallest ones I’m seeing on ollama are > 100GB e.g. https://ollama.com/library/qwen3.8-flash-next/tags
Re:Cerebras Deploys Trillion-Parameter AI Models by sgsjchs on Sunday October 25, 2026 @01:22PM (#50233848)
This arch uses engrams aka per layer embeddings. Not all of those GBs need to go in vram.
Re:Cerebras Deploys Trillion-Parameter AI Models by cyanydeez on Monday October 26, 2026 @08:24AM (#50242479)
Halogen flash server for strix halo with quant 4. Its offloading the PLE.
Re:Cerebras Deploys Trillion-Parameter AI Models by adastra22 on Sunday October 25, 2026 @02:48AM (#50228965)
GPT-5.6 Sol Ultrafast is running on Cerebras hardware.
Re:Cerebras Deploys Trillion-Parameter AI Models by keyle on Sunday October 25, 2026 @03:05AM (#50229036)
It is not available to "developers".
Re:Cerebras Deploys Trillion-Parameter AI Models by madaxe_again on Sunday October 25, 2026 @03:09AM (#50229062)
Sure it is, you just have to pay for it. TANSTAAFL.
Re:Cerebras Deploys Trillion-Parameter AI Models by Topfi on Sunday October 25, 2026 @04:59AM (#50229580)
That has never been made available even to limited customers as far as reporting goes. Currently available ultrafast on GPT-6 Astra and GPT-6.1 Sol are all using Nvidia for good reason [0].

The fact that Cerebras, which has a willing and major customer like OpenAI begging at their heels to deploy on their hardware, now posts about a 2.8T open weight model says a lot.

[0] https://blogs.nvidia.com/blog/gpus-openai-gpt-6-astra-ultraf...

Re:Cerebras Deploys Trillion-Parameter AI Models by chrismustcode on Sunday October 25, 2026 @05:38AM (#50229756)
You can get 6.1 sol on ultrafast.

For some reason though on openrouter the tps of 6.1 sol ultrafast is usually lower than opus 5.5 fast. Is there readings off as surely cerebras hardware is faster than low user density nvidia still?

https://openrouter.ai/openai/gpt-6.1-sol

Re:Cerebras Deploys Trillion-Parameter AI Models by Topfi on Sunday October 25, 2026 @05:47AM (#50229807)
GPT-6.1 Sol Ultrafast also uses Nvidia hardware, not Cerebras, just far better provisioned vs how they usually provide inference allowing for higher throughput.
Re:Cerebras Deploys Trillion-Parameter AI Models by stingraycharles on Sunday October 25, 2026 @08:20AM (#50230794)
So then how is OpenAI using Cerebras? Or are they not?

Or, from a different perspective: who is using Cerebras for what purpose?

Re:Cerebras Deploys Trillion-Parameter AI Models by davidweatherall on Sunday October 25, 2026 @08:27AM (#50230837)
Jane Street bought up all of the Cerebras x OpenAI compute I believe. Couldn't find a great source, this tweet has a good few replies citing other sources as well https://x.com/cryptopunk7213/status/2106834536868856223
Re:Cerebras Deploys Trillion-Parameter AI Models by stingraycharles on Sunday October 25, 2026 @09:39AM (#50231473)
Oh that would make a lot of sense, basically supply limited.
Re:Cerebras Deploys Trillion-Parameter AI Models by fastball on Sunday October 25, 2026 @03:02PM (#50234933)
"Much better provisioned" might be overselling it. Mainly they are just serving very few customers per Nvidia GPU, which is an expensive solution to the problem.
Re:Cerebras Deploys Trillion-Parameter AI Models by preommr on Sunday October 25, 2026 @10:25AM (#50231935)
> GPT OSS 120b

It's extremely frustrating that these fast models are very old at this point. Chat jimmy launched in february, and nothing has come of it, not even GA let alone improvements with intelligence.

AI is currently moving so fast, that might as well be a lifetime.

We get SOTA from anthropic/openai every few weeks on models that seem to focus on one useful thing (e.g. opus) and then 5 useless things like booking restuarants. We're ignoring really useful applications for some reason.

It took jev to shake things up. I remember seeing reactions that were basically "people want this? the big labs probably have some version of this lying around"... and it's like yea, maybe these labs need to fix their priorities a little bit.

Re:Cerebras Deploys Trillion-Parameter AI Models by p1esk on Sunday October 25, 2026 @11:19AM (#50232509)
what might be useful to you (e.g. coding) might be completely useless to 99% of the people
Re:Cerebras Deploys Trillion-Parameter AI Models by halJordan on Sunday October 25, 2026 @11:48AM (#50232843)
Chatjimmy got taalas aquired
Re:Cerebras Deploys Trillion-Parameter AI Models by froh on Sunday October 25, 2026 @01:06PM (#50233707)
yes, and by amd which announced to integrate it with their hardware.
Re:Cerebras Deploys Trillion-Parameter AI Models by everforward on Sunday October 25, 2026 @12:50PM (#50233532)
I suspect they’re ignoring it because there isn’t money in it.

Jev is super cool, and it’s super cool that it can handle novel categorization. Most organizations aren’t spending the big bucks on novel classification, though. They’re doing the same thing a billion times, which means Jev has to compete with having a custom model built and doing your own inference.

It’s very hard to build something generic like Jev, but it’s fairly manageable to build something more specific that handles a single kind of task. Especially at the scale where OpenAI would care about your attention.

I don’t really understand neural networks and I managed to train one to do hand gesture recognition on an old Leap Motion. It’s a pretty limited domain, but it trains on the CPU of my decade old laptop in under 3 minutes. The accuracy is quite good too, especially when you discount errors from the Leap (like misidentifying which finger is extended).

Re:Cerebras Deploys Trillion-Parameter AI Models by Miraste on Sunday October 25, 2026 @01:44PM (#50234045)
When talking about Jev, people love to bring up the whole "bitter lesson" idea that generalization will win out. Maybe it will in the end, but I tried most of the recent Jev-likes on my own not-very-complex use case and they weren't competitive with domain-specific classification at all.
Re:Cerebras Deploys Trillion-Parameter AI Models by weitendorf on Sunday October 25, 2026 @02:11PM (#50234332)
General classifiers are actually incredible for two things:

1. inference on arbitrary context / world knowledge with fast, structured output

2. I almost don't want to say it because I don't know if people realize this yet... but that makes traces from any kind of non-AI process or system "learnable", you can easily model it by by just asking it to predict the next outcome during learning and compare against what happened.

which also means it could also learn new strategies through rollouts in a way that generalizes across semantic space which means...

so these models are actually quite promisingly useful for existing companies because they do have general traces of what happened in their company and want to discover ways they can tweak their business processes, and then there is durable demand for models you can embed within a product or UX flow without exposing your user to slop + then train or customize and build real IP or custom capabilities

general classification itself is quite interesting IMO for applications that need to make judgment calls or select one of many UX actions, and a much better fit for this than LLMs. So if you think about it, this is the type of thing businesses might be calling a lot - every trace (eg user session, package shipment, sales motion, marketing campaign, job outreach) + proactively throughout dynamic experiences to decide whether to eg end a call or which of multiple UIs to manipulate

this is just an unfamiliar product experience to most people because it hasn't really been bootstrappable with a strong general baseline until now. But now it's posible and I think it will be very col.

Re:Cerebras Deploys Trillion-Parameter AI Models by everforward on Sunday October 25, 2026 @04:38PM (#50235831)
I think you’re misunderstanding me; I’m not saying places will use an LLM instead (whether cloud-based or on-premise).

I’m talking about them training their own models with a much more specific purpose.

You brought up choosing a menu option in a UI. Jev is really good at something like “find the menu option that best fits the users prompt”. That’s general, and requires a big complicated model.

Companies are usually more interested in something less variable. Ie the menu option they’re looking for is fixed. “Take an arbitrary website and find the Login button”. That’s a _way_ simpler model, and well within the abilities of many large companies to train.

You only need to pay Jev if there are so many variables the model basically has to be general.

Eg classifiers are a separate thing from LLMs and are literally only capable of structured output. It will output one of the X options it was trained to recognize.

I think some of this is that the Venn diagram of people who like AI and the people who like machine learning isn’t a circle. There’s a lot of “intelligences” out there that aren’t an LLM or decision model.

Re:Cerebras Deploys Trillion-Parameter AI Models by catlifeonmars on Sunday October 25, 2026 @05:09PM (#50236087)
I don’t think I really understand the motivation to move as fast as possible. Who cares if you’re behind by 6 months? Objectively you’re still ahead of where you would be without the 6 months old tech.
Re:Cerebras Deploys Trillion-Parameter AI Models by nexus2045 on Sunday October 25, 2026 @11:53AM (#50232905)
This - I use Cerebras supported models here and there, the speed is amazing and I can see the potential with having almost instant inference, but the models are so outdated at this point.

Cerebras if you're reading this, get your act together - maybe that will also help your stock price. Give consumers more options, not just to OpenAI.

Re:Cerebras Deploys Trillion-Parameter AI Models by splicebot on Sunday October 25, 2026 @12:08PM (#50233099)
Cerebras isn't public and are almost certainly going to exit by being acquired. So demos to the public don't matter.
Re:Cerebras Deploys Trillion-Parameter AI Models by hijinks on Sunday October 25, 2026 @12:19PM (#50233197)
they are public $CBRS
Re:Cerebras Deploys Trillion-Parameter AI Models by adityamwagh on Sunday October 25, 2026 @12:50PM (#50233537)
Re:Cerebras Deploys Trillion-Parameter AI Models by jubilanti on Monday October 26, 2026 @09:21AM (#50243186)
I'm sure they were planning on running old models forever, but now that they've read your comment, they'll be motivated to get back to work.

Thanks for your contribution to the discussion.