The large models are actually MoE these days so they're usable on ordinary hardware with weights streaming from SSD, just very slow. You're nonethess right that it makes the cloud-based offering more popular, since you can use that for convenience after testing a few inferences locally.

“Usable but slow” is how you could run regardless of MoE or not, the model architecture has nothing to do with it. MoE might run N times faster than non MoE where N is the number of experts but that’s it.

There are plenty of model providers that can serve them though at cheaper prices and cannibalize Alibaba revenue.

You can run it on runpod for example.

I'm running qwen 3.5 397b on very standard hardware. Just use the unsloth quants, they're great. I get like 20t/s or something.

It's super not a publicity stunt, qwen 3.5 is the base of the best local models out there IMO.

Well, you didn’t post the specs on your rig. I think it’s probably more correct to say that you run it on very beefy but readily available hardware. My point was not that nobody could run a 300B model, but rather that a 300B model is not going to be runnable by a majority of people. Sure, anyone who wants to run that model and has the money to purchase the hardware can do it. But the hardware is going to be pricey and most people don’t already have it unless they were trying to run large models before this. My overarching point is that most people with average laptop specs purchased over the last 3 to 5 years are going to have to consume this from the cloud. Which is great for Qwen.

I can run a 300b model, but I don't do it. We need the H100's for training

I just have a 3090 and 64gb ram. Yes this is more than most people have, but calling it a "publicity stunt" is just so uncharitably weird of a characterization.

There's smaller models all the way down too.

Like this should be _exactly_ what we want companies to release.

I apologize. I didn’t mean to suggest a “publicity stunt” was a negative. Perhaps I should have said that it was a great marketing strategy. My point was, they can cite all the metrics associated with a frontier model and yet to actually get those metrics most users will have to purchase cloud-based services. That all. And sure, some people will definitely be able to run the model and benefit from it. As you say, this is what we want.

Yeah it is apparently some kind of marketing strategy I guess. Tbh I can't imagine they're getting enough out of it for it to make sense for them. Personally, I'm not looking the gift horse in the mouth too closely, I'm just happy that the current insane rush to make better models means we get some decent "open" ones to play with.