I think we'll eventually hit an information theoretic type of wall with physical hardware and GPUs and need a similar AI breakthrough as well as the development refinement of logical/physical qubits in the quantum computing space with some analogue to the transformer architecture to continue accelerating. However, I think there must be many years of development and refinement that can take place before that paradigm shift to overcome the physical compute wall is necessary. This is just my theory, but I'm young enough that I'm expecting with the rate that we are advancing, I will see AI / LLM analogues developed and run on a quantum computer in my lifetime.

I think the bottleneck will be the current one.

Fabs.

Either needing more fabs, new types of fabs, retooling existing fabs.

All of that takes years.

maybe we can design our way out of that too. But, I suppose that would be the similar breakthrough you are mentioning.

Eventually the heat death of the universe will come, so clearly any prediction made needs some kind of time frame attached.

I think it's fully possible that it continues being exponential for decades like Moore's law did (and still is depending on exactly what you measure)