> Do we still think we'll have soft take off?

There's still no evidence we'll have any take off. At least in the "Foom!" sense of LLMs independently improving themselves iteratively to substantial new levels being reliably sustained over many generations.

To be clear, I think LLMs are valuable and will continue to significantly improve. But self-sustaining runaway positive feedback loops delivering exponential improvements resulting in leaps of tangible, real-world utility is a substantially different hypothesis. All the impressive and rapid achievements in LLMs to date can still be true while major elements required for Foom-ish exponential take-off are still missing.

Yes, but also you'll never have any early evidence of the Foom until the Foom itself happens.

If only General Relativity had such an ironclad defense of being as unfalsifiable as Foom Hypothesis is. We could’ve avoided all of the quantum physics nonsense.

it doesn't mean it's unfalsifiable - it's a prediction about the future so you can falsify it when there's a bound on when it is going to happen. it just means there's little to no warning. I think it's a significant risk to AI progress that it can reach some sort of improvement speed > speed of warning or any threats from AI improvement

https://zenodo.org/records/18498514 GR is nonsense!

To me FOOM means like the hardest of hard takeoffs and improving at a sustained rate which is higher than without humans is not a takeoff at all.

Exponential growth may look like a very slow increase at first, but it's still exponential growth.

Sigmoids may look like exponential growth at first, until they saturate. Early growth alone cannot distinguish between them.

Intelligence must be sigmoid of course, but it may not saturate until well past human intelligence.

Intelligence might be more like an optimization problem, fitting inputs to optimal outputs. Sometimes reality is simply too chaotic to model precisely so there is a limit to how good that optimization can be.

It would be like distance to the top of a mountain. Even if someone is 10x closer, they could still only be within arms reach.

If it's exponential growth. It may just as well be some slow growth and continue to be so.

On the other hand: Perception of change might not be linear but logarithmic.

(= it might take an order of magnitude of improvements to be perceived as a substantial upgrade)

So the perceived rate of change might be linear.

It's definitely true for some things such as wealth:

- $2000 is a lot of you have $1000.

- It's a substantial improvement of you have $10000.

- It's not a lot you have $1m

- It does not matter if you have $1b

$2000 is not substantial over $1b on the linear scale

2k is the same on the linear scale no matter where you are. that's what the linear scale is about.

you're already interpreting this on the log scale

I'm only saying no to keep optimistic tbh

It feels crazy to just say we might see a fundamental shift in 5 years.

But the current addition to compute and research etc. def goes in this direction I think.

making the specifications is still hard, and checking how well results match against specifications is still hard.

i dont think the model will figure that out on its own, because the human in the loop is the verification method for saying if its doing better or not, and more importantly, defining better

I think the limiting factor is capital, not code. And I doubt GPTX is anymore competent at raising funds than the other, fleshy, snake oilers...

This has already been going on for years. It's just that they were using GPT 4.5 to work on GPT 5. All this announcement mean is that they're confident enough in early GPT 5.3 model output to further refine GPT 5.3 based on initial 5.3. But yes, takeoff will still happen because of this recursive self improvement works, it's just that we're already past the inception point.

I can't tell if this is a serious conversation anymore.

“Best start believing in science fiction stories. You're in one.”

https://x.com/TheZvi/status/2017310187309113781

I think it's important in AI discussions to reason correctly from fundamentals and not disregard possibilities simply because they seem like fiction/absurd. If the reasoning is sound, it could well happen.

I totally got what you felt there. We are truly living in a sci-fi world

I guess humans were involved in all that, so how is that anything but tool use?