You really need to let your priors go if you still use this tired trope of next token prediction. It’s as useful for discussion as saying that human brain is made of fat, protein and carbohydrates - yeah that’s true, but it’s useless observation.

I think the “next token prediction” is too dismissive and reductive a framing of their capabilities at this point.

Yes we all know that’s what they do, and guns just push a few grams of lead out of a pipe. It’s what you can do with that capability that is important.

When you couldn’t count the R’s in strawberry it would have been a more effective statement. But a few short years later they are being used to solve millennium puzzles.

What if the scaling continues? A model n years from now gets burned into silicon, a single company has millions of the chips, and in a few moments the system spend more time “thinking” than humans have ever spent thinking collectively?

If it’s even possible I don’t think there’s anything we can do about it at this point. Cat’s out of the bag.