The AI doomer speculation is of the form "if things continue improving the way they are currently improving, there is a non-zero risk that we cause human extinction." We have battle-tested rules indicating that increasing data size and compute will lower perplexity on the corpora. This is not for sure, but it is likely given inductive assumptions that things generally keep behaving the same way.

What we don't know is what qualitative capabilities are unlocked at different perplexity levels.

I think this is a lot less speculative than "things will not continue improving the way they are currently improving."