I think we’re no where near a physical information theoretic limit.
The hardware also is, so there ought to be a whole lot more room for improvement.
“ RSI might pick all of the low hanging fruit soon”
Scam Altman is that you?
It is just occurring to me that “RSI” expands to recursive self improvement. Thought people were talking about repetitive stress injuries; either in regards to programmers writing too much code/not having to write code anymore, or the frontier AI companies and their tendency to applaud themselves.
There are going to still be worthwhile improvements but they are going to be more like not how to make transformers 10x cheaper but how to make next training run cost 9 trillions instead of 10 with a very particular optimization designed at the cost of hundreds of millions for this one specific run.