He distinguishes between weak recursion (using sotfware to build more software), and strong recursion (handing over model development to AI which does everything), and makes a pretty strong case for the fact that only weak recursion is happening.

This maps to my experience with AI models, they reduce the cost of experimentation massively, but they can't (currently) design experiments themselves.

Even then, training RL is very data hungry, and most profitable tasks don't have good data so even if we got superintelligence (whatever that means), it's entirely possible that progress would still be bottlenecked by lack of high quality training data.

The hope was that weak recursion could produce strong recursion by automating the iteration of ML pipelines.

Theres also the idea of LLMs producing synthetic data to contribute to new models. Perhaps that is not as effective as hoped.

This never made sense to me. Seems analogus to inbreeding in biological organisms.

More complex synthetic data generates things that could be in the training data but aren't for some reason. For example, machine translation training corpuses often contain a lot of formal language (because formal documents are the ones that get translated), so you might generate a bunch of casual conversation to improve the translation model's ability to handle that.

Wasn't model collapse/model decay a big issue since the beginning? i know that RL kinda mitigate the effect, but if this was close to not be an issue anymore, wouldn't we have a paper on that already? All that i've seen on the subject was about mitigation, never about solving the issue.

Sorry if my question is dumb, I need to do some research about the difference between image generator and text generator, i feel like my understanding of LLM is skewed by my usage and understanding of Z-turbo, but i only have that much time in my week.

AI can be a "tool", but only so long as you understand how it works and what the implications of its use are. Being an enormously powerful tool implies, the consequences of not understanding and not knowing are proportionally enormous as well.

Currently, LLMs are used practically exclusively by "professionals" who, more or less, know what they're doing. The idea of applying them "everywhere" completely misses the gap in understanding and responsibility for the vast majority.

The consequences of "AI use" are manifold and very poorly understood. That doesn't make them less impactful, on the contrary.

Whether you look at "humanoid robots" or incorporeal LLMs, when they are more capable than yourself, you become their dependent counterpart, not their "master".

https://www.tradingkey.com/analysis/stocks/us-stocks/2623001...

Anthropic desribes itself as a "public benefit corporation" in its IPO. The irony. Or is it sarcasm?

A god isn't your subordinate. You cannot "program" your wishes into it, like a genie/jinn. They (or their investors) want to make themselves into gods. Over you.

Not Anthropic specifically, but there's this thing that I recently read:

https://www.nytimes.com/2026/11/09/opinion/ai-anthropic-doom...:

> A.I. company leaders often talk about how their work might lead to human extinction soon. This is astonishing. Even if you wanted to destroy humanity, you wouldn’t say so. If, however, you wanted to save the human species, wouldn’t you stop developing this technology?

> There is an explanation for why some tech leaders, so seemingly worried about an extinction-level event, are plunging ahead with A.I. development: Mankind’s survival, while important to many of them, is less critical than building something they believe to be not only conscious but godlike. A substantial portion of the A.I. community, in celebrating this belief, is quick to support any action or statement that reinforces the idea that the technology is like a deity. They might call for regulation, but only in specific ways — perhaps by deploying more A.I. to control A.I. — that adore the technology’s powers. It is that adoration that is nonnegotiable.

It's by the guy who apparently invented VR and was apparently mentored by Marvin Minsky. So he likely has the kind of access that gives his opinions credibility.

I think it's unlikely that an AI enthusiasts would literally claim "they are building a god," but such an attitude (or something very similar) is implied by their statements and attitudes, e.g.

1. Talk/hope of building something that so exceeds human capability, it's beyond human comprehension and control.

2. Taking existential risks for humanity (by their own statements) to try to get there, against all reason.

3. etc.

I guess they think AI will become smarter than people but I'm not sure that's a god.

This was always the case with ML, even absent RSI. Big bag of weights is a pain to figure out exactly why it can do whatever it is that it does, leading to all kinds of blind spots about how they go wrong.

> Currently, LLMs are used practically exclusively by "professionals" who, more or less, know what they're doing.

Even the people who made them don't really know what they're doing, hence https://www.felonybench.com

And what is that mission?

I'd almost prefer it if they were motivated by money, then at least we know what we're in for. All this 'mission' stuff seems to effectively amount to 'we're saving the planet, sorry you got in our way'.

sniffles