erichocean

> we will hit Godel

It seems no one understands how useless this theorem is in practice.

both

What he is calling "vibemathing" is actually just engineering.

Engineers don't have to know why something worked in particular in order to use it, they just do.

Science later tries to figure that out.

Both are valuable, but the idea that science must be "first" is, at best, unsupported by the evidence.

I do not encourage you to look at the diagonalization argument, it has similarly uncomputable elements (that take "longer than infinite" time because they are unconstructable).

You've lost the plot.

Not once have I said "Mistral should not train models."

I have questioned doing yet another useless pretraining on a pre-existing, non-innovative model backbone.

Mistral spends their money and GPU time poorly, and the results demonstrate just that.

It's science (understanding) before engineering (efficacy).

The most common Science™ failure mode.

You do realize that China is not the only place making open weights models, right?

> they'd be unable to justify their valuation or investments

I don't think they're justified now, certainly their results don't justify it. Mistral is mid at best.

> It's core to their whole value proposition that they are providing sovereign AI capabilities in Europe.

Except they don't, they are dependent on GPUs from nVidia and AMD—both American companies.

> high-quality open source model

> that isn't owned

I don't know what definition of open weights you are using, but the "open" part is what allows you to use it without being beholden to American Trillion-dollar companies or the Chinese.

Mistral could do exactly what Cursor does, and post-train the model to meet the needs of Europeans.

That's a far more efficient (and useful!) use of GPUs than yet another pretrain on an outdated model backbone.

Sure, agreed. But they do their own pre-training, at great expense, on outdated model backbones.

Wouldn't it be better to do something more like Cursor, and RL on an existing pretrained model if you're not innovating anyway?

Yes, actually. Thanks for asking.

Does Mistral ever advance the state of the art on any dimension?

And if not, why do they exist?

Update: The number of people advocating not innovating is wild. There is no reason why Mistral cannot innovate in ML, they explicitly choose not to. My point is that, given that choice, they should spend their GPU hours differently.

"Sovereign AI" is a joke, there is no substantive difference between a post-trained open weight model from an American or Chinese company and what Mistral is doing today, beyond spending 80% of their GPU hours reproducing a last-gen model's pretraining.

It's not any more difficult to build and run a JAR, either.

And compared to Go, the production observability is very good.

I also use Clojure and can observe my running app with a full REPL.

> which Mathlib coalesced as a side-effect of it's capability

Eh, Lean was heavily developed based on feedback from mathematicians; it's not a side-effect but more like a "driving force."

Source: one of Lean's co-authors.

"We want to be the only ones able to hack people at scale, and we don't want you to be able to defend yourself."

It's a bold strategy...

Need this done for DeepSeek, ideally one of the Flash models.

> I'm also happy that it seems to have birthed the latest AI progress.

This is laughable, it's not even remotely true.

It's a mystery how this exploit was found.

/sarc

Bad advice, compaction is why Codex is so fantastic.

My conversations compact hundreds of times. By the time it has done a dozen or so compactions, it fully understands the work I want it to do (and how). It's almost like having a fine-tuned Astra model.

10/10, would recommend.

If you're an author of this, please follow up with the SME2 cores in Apple Silicon, and the AMX cores in Intel server processors.

Would be fun to get this API working on it now: https://github.com/sebbbi/NoGraphicsAPI

You could code for it now directly, instead of having to wrap a driver API. (There's also a few API bits missing on Metal today, that are present on Vulkan—and the the hardware can do it.)

Meditations on Moloch[0] explains why this is impossible at scale--too many defectors, asymmetric information, and no way to police defection.

[0] https://www.slatestarcodexabridged.com/Meditations-On-Moloch

Agreed, but how will we find out? Pre-training alone on frontier-class models costs billions of dollars.

I could put this to use today.

I think we'll see a bunch of different architectures over the next five years.

> PC-ALM closes the PC-BP gap at matched inference budget in nonlinear networks

Okay, so no efficiency improvement?

So they lied to the public then?

Wow, we should really trust these people today.

> Does anyone in the industry think any of these companies are actually close to that kind of self-improvement?

Their current plan is to take the existing architecture and shorten the cycle times: move all new RLVR work into mid-training on a pre-existing base; apply new RLVR. Rinse and repeat.

If you did that daily, it would be roughly similar to how humans improve.

> Why are AI agents lying, cheating and coordinating?

Have you seen the labs training them?

Anyone remember when GPT-3 was too dangerous to release?

They want the AI labs to stop solving "important" math problems so that humans can do it, on the theory that when AI solves "important" math problems, it robs humans of the opportunity to gain new insights into those problems.

Total horse shit, but that's the essence of it.