For AI research groups: funding (venture capital, mostly) -> hardware that can be bought -> amount of compute that can be done. Efficiency improvements likely means: 'heavier' models deployed.

For end-user applications: as determined by that applications' constraints. Like TDP or battery life on a smartphone. Or (for subscription based online services): there's a price tag attached, and someone picks up the bill.

So, long-term it'll probably depend on how many useful applications are developed. If changing society completely, then sure expect a continuing stream of $$ to be thrown at the required compute. If mostly a hype, that $$ will quickly dry up.