> Haven’t you ever had the experience of knowing exactly what you want to say but struggling to find the word for it?

I came here to suggest this exact thing. Humans clearly have some understanding of the meaning behind words and sentences. However, I think it's wrong, an overgeneralization, to suggest that ChatGPT is just statistically predicting the next word. While that may technically be true, I think buried deep within it has ways of modeling/encoding common concepts and ideas, maybe similar to how a human models concepts and ideas in their mind.

Then there's the whole problem of consciousness, but I won't get into that here.

>I think buried deep within it has ways of modeling/encoding common concepts and ideas, maybe similar to how a human models concepts and ideas in their mind

Right, maybe meaning and concepts are just “things that usually go together” with different levels of distance.

We’re comparing our minds, which we know very little of, to LLM, which are blackboxes. It seems like a lot of certainty in face of such vast ignorance.

> Haven’t you ever had the experience of knowing exactly what you want to say but struggling to find the word for it? How could that be if we’re just randomly selecting words like a souped-up Markov chain?

All words in the normal cache have a low probability, so you have to hit disk to get the full word list.

We’re not randomly selecting but probabilistically predicting. Haven’t you noticed you predict the next word people will say with very high [________]?

Within a sentence sure. And often arguments are familiar enough that even an essay I’ll know where it’s going. But we do occasionally have original ideas.

I don’t think we know what original really is to make a meaningful distinction between derivative work.

With enough mixing and matching and a tiny bit of randomness, I bet a lot of original thoughts could be created.

These things are hard to quantify but ChatGPT writing does have a certain anodyne character, in my mind, when I read it. Then again, I can't quantify it, I could be biased by my foreknowledge of the writing's provenance, and even if I'm right, the world is full of situations where bland writing is suitable or even desired.

It absolutely does. It’s wishy-washy nature is harder to disagree with, sounds more credible, etc.

Sandwiches!