chrsw

More like, sometimes when you publish slop it gets called out as slop.

I upvoted without looking into the details because it sounds cool, whether it's new or not.

I suspect a lot of upvotes also come from people afraid of closed western labs running away with AI. Similar to other articles with positive news about open weight models.

I'm pretty sure I heard Chen say on a podcast that he has a significant backlog of posts that he puts up when he feels like it.

The modern internet is a largely centralized network built for the following purposes:

- commerce

- surveillance

- platforming voices for states and the wealthy

I see. So maybe the situation isn’t as bad as I thought. I was thinking if you took all the money and resources spent on building the biggest models, across the two big labs, the hyperscalers, the dozens of neo labs, and you reallocated that money to building value on cheaper Chinese models that would be a much better proposition than doing whatever it takes to be head of Chinese model performance by a few percentage points for a few months. And I wasn’t just referring to the cost of training one model, I was considering the total cost of maintaining an entire ecosystem with the ability to do this.

But what I’m hearing is this is a negligible cost compared to inference buildouts. Fair enough.

Maybe I'm conflating. I could be dead wrong about all this. But I'm trying to zoom out and look at the big picture. So, I'm letting the details sort of blend away and I'm just asking myself: "Is such a large financial sink in our economy, at the macro level, really going to offer the value that the investment warrants?". So, the particulars of who exactly is spending what where I guess doesn't concern me as much. What I'm thinking about is American companies, American people, American dollars going to something that seems like an iffy value proposition.

Now, if you tell me American businesses are turning cheap Chinese tokens into goods and services of high value, for Americans and people all over the world, then obviously I would have a different outlook on what's happening here.

So, the US economy spends trillions of dollars to be at the "frontier" and China puts out a model that basically does the same thing, for free, and for a tiny fraction of the development cost. How is this sustainable?

My prediction is that in in a few years, maybe more, the current models will be seen as so primitive that they're not worth mentioning outside of a historical context.

I can think of plenty of tasks that even the closed frontier models fail miserably at. From my perspective we have a long way to go for these models to get better.

There are bills to pay. Who has time for friends?

I don't understand how people are so certain on how this will play out. We have people basically saying companies like Anthropic offer no value because open source models are free. Then we have others saying once frontier AI labs go RSI FOOM takeoff they'll essentially _become_ the tech industry.

None of this sounds credible.

A hybrid approach might work better for robotics. A model-in-silicon for body control and sensing, then a different model on processors for reasoning, speech and decision making.

No, there still plenty of tasks that LLMs can’t do well or at all. Yes, we should absolutely make models cheaper but there’s a long way to go until they’re “good enough”. And I’m talking about the frontier closed models too, not just the open ones. So it’s not an “instead of” position we’re in. It’s both.

Imagine if a car company like Ford or Toyota came out and said their new cars are kind of out of control and they needed special indemnities and protection from liabilities if they want to keep selling cars.

You can use open weights models right now and ignore all this fuss. And by the time open weights models are objectively and subjectively on par with what Fable/Astra are today, the frontier labs will probably offer something newer, more capable and more expensive. The models OpenAI and Anthropic roll out clearly aren’t for people on a strict budget.

Maybe one day open weights and closed models will hit the same wall?

Would a 100% compatible and open source CUDA stack be “competition” too?

We need to immediately set the precedent that ultimately humans and companies are responsible for what their AI systems do.

Interesting.

And I also found the video I was referring to https://www.youtube.com/watch?v=FHQfmJEpRmU

I heard someone who studies this sort of thing say basically what biological neurons are trying to do is predict as well. Predicting what exactly? I’m not sure. The next time they should fire or something. I can’t find the YouTube video now.

I don’t think that’s what’s going on. I notice flaws on day one of model releases. But I also notice improvements if the model is truly more advanced than what I’m used to. Then over time the same questions or tasks return worse results.

What is actually stopping these model companies from running a model at full capacity on release then once its name rings out, start serving users quantized garbage?

600 sq ft is about the size of my apartment, which at market rate is about $3k/mo.

Interesting approach. I would like to see a benchmark like this without web or mobile app code. In fact just: systems code (operating systems, drivers, low level applications), libraries, native apps, firmware, robotics software, signal processing, embedded code, bare-metal, RTOS, hardware design languages, EDA tools, etc.

That says more about the dire state of the industry than Apple’s own engineering preferences and their brand optics.

I agree we’d switch to it. I guess what I’m doubting is if a company can recover from that sort of stumble in the first place. And they might not want to either. They might think there’s more value somewhere else besides trying to get back to the absolute performance and capability frontier. Smaller models targeted to specific domains that large models would be too inefficient at no matter how large they get or how clever you are at distillation, for example.

I can call a coworker right now and have a real time conversation with them without feeling like I’m talking to a frustrating machine. Most of them, anyway. But I guess that’s “moving the goalposts”.

Do labs come back from disasters like GDM’s 3.5 pretrain? I am thinking of Meta’s Llama 4. Meta is just now starting to be taken seriously again but they are definitely not at the frontier. And when I say “come back” I mean have an Opus 4.5 moment, which was really mind blowing for me at the time. Fable was a similar leap, just not as big.

Where I’m from $25k is the down payment on a parking space in a garage, not an entire house (or condo).

Only the Chinese authorities can stop Chinese labs from distilling from western labs. And they won’t do that, for obvious reasons.

At some point it won’t be realistic to ask a human to solve a technical problem any more. I’m not suggesting it will happen soon but I can at least imagine that day coming. Whereas before modern “AI”, that possibility wasn’t even on my radar.