That's not how the tokenizer works. A novel word like "platoggle" is decomposed into three separate tokens, "pl", "at", and "oggle". You can see for yourself how prompts are tokenized: https://platform.openai.com/tokenizer
If that is a inference time performance or training time performance or a model size issue or just total nonsense, I wouldn't know.
So notice it doesn't contain any vowels, since almost all consonant-vowel pairs are sufficiently frequent in the training text as to be tokenised at least as a pair. E.g. "guq" is tokenised as "gu" + "q", since "gu" is common enough.
(Compare "gun" which is just tokenised as a single token "gun", as it's common enough in the training set as a word on its own, so it doesn't need to tokenise it as "gu"+"n".)
The only exceptions I found with consonant-vowel pairs being tokenised as pairs were ones like "qe", tokenised as "q" + "e". Or "qo" as "q"+"o". Which I guess makes sense, given these will be low-frequency pairings in the training text -- compare "qu" just tokenised as "qu".
(Though I didn't test all consonant-vowel pairs, so there may be more).
Spell the longest state name backwards, omitting one t of your choice
> The longest state name is "Massachusetts." Omitting one "t" and spelling it backwards, we get "sttesuhcassaM."
I wouldn't have it quit it's day job yet.
It's not great at random substring manipulation, but spelling words is so common it almost certainly has internalized some structure akin to a programmatic representation of decomposing a string into characters.
The biggest difference I see is that it doesn’t have consciousness: the capacity to detach it’s simulation of the world of the present.
When we are conscious we jump in a simulation in our head we imagine a future where we do one thing and then we get one kind of result. If we don’t like the results we have in our simulation we jump to an alternative and so on until we find a path that we like.
We can attach ours simulation to past memories and again simulate alternative, to try to integrate what happened to us. GPT doesn’t seem to have that yet, it’s always in the present moment, like a dog, no worry for the future.
Me: I would like to play a would game with you. I'm going to tell you a made-up word, and then ask you a question about it. Ready?
ChatGPT: Sure, I'm ready to play! Let's get started.
Me: Does the made-up word plukatoon have the letter e in it?
ChatGPT: No, the made-up word "plukatoon" does not have the letter "e" in it.
-- Note, at this point I was really surprised it got it right. I chose "e" thinking that it would default to yes given the popularity of e.
Me: Correct! Does the made-up work ziddlebum have the letter e in it?
ChatGPT: No, the made-up word "ziddlebum" also does not have the letter "e" in it.
-- OK, proof that it's "not perfect" as you point out
Me: Does the made-up word folladel have the letter o in it?
ChatGPT: Yes, the made-up word "folladel" does have the letter "o" in it.
-- Edit, and finally:
Me: How do you spell the made up word "platoggle"?
ChatGPT: The made-up word "platoggle" is spelled as follows: P-L-A-T-O-G-G-L-E.
OK, so that last one TOTALLY blew my mind. I'm guessing it's never seen the word "platoggle" before, so even if it learned previous spelling associations with other tokens, it couldn't possibly understand how to spell "platoggle" if it's treating it just as a single, never-before-seen, opaque token, could it?
The more I play with this stuff the more existential crises I have.