The last few iterations show a logarithmic curve at best tbh. If we are to see a major advance, it'll be something like the implementation and infrastructure for byte-level transformers.