I think you're discounting efficiency gains — through a series of individually minor breakthroughs in LLM tech I think we could end up with things like 100M+ token context windows

We've already seen this sort of incrementalism over the past couple of years, the initial buzz started without much more than a 2048 context window and we're seeing models with 1M out there now that are significantly more capable.