Qwen 3 is coming imminently as well https://github.com/huggingface/transformers/pull/36878 and it feels like Llama 4 should be coming in the next month or so.

That said none of the recent string of releases has done much yet to "smash a wall", they've just met the larger proprietary models where they already were. I'm hoping R2 or the like really changes that by showing ChatGPT 3->3.5 or 3.5->4 level generational jumps are still possible beyond the current state of the art, not just beyond current models of a given size.

> met the larger proprietary models where they already were

This is smashing the wall.

Also if you just care about breaking absolute numbers, OpenAI released 4.5 a month back which is SOTA in base model, planning to release O3 full in maybe a month, and Deepseek released new V3 which is again SOTA in many aspects.

Google's announcements are mostly vaporware anyway. Btw, where is Gemini Ultra 1? how about Gemini Ultra 2?

It is already on the LLM arena right, codename nebula? But you are right they can fuck up their releases royally.

I guess they don’t do ultras anymore, but where was the announcement for it? What other announcement was vaporware?

Any more info on the new Google model?

> We haven't hit the wall yet.

The models are iterative improvements, but I haven't seen night and day differences since GPT3 and 3.5

Yeah. Scaling up pretraining and huge models appears to be done. But I think we're still advancing the frontier in the other direction -- i.e., how much capability and knowledge can we cram into smaller and smaller models?

Tbh such a big jump from current capability would be ASI already

Because 3.5 has a new capability which is following instructions. Right now we are in 3.5 range in conversation AI and native image generation, both of which feels magical.

We've slid into the upper S curve though.

Asymptotic improvement will never hit the wall