The pallette for LLM is tokens not characters.
Re:Visual ChatGPT
by dragonwriter on Friday March 10, 2023 @10:32AM
(#35095634)
And if it is characters (as it is for some models), its more than 26 of them for English. Between space, case, punctuation, and digits, its basically 7-bit ASCII without most of the control characters (newline is semantically important, the rest not), almost 100 characters.
The number of tokens is more in the range of 40k to 250k tokens.
Right.
Whereas for pixels there are over 16 million.
How many more than 250k is that?
Over 16 million.
More like 256 of them, just an image is actually 3 layers with RGB. If you are going into combinations, I could easily say that triplets of tokens are actually what's important, and so 250k^3 is the real number of whatever.
Re:Visual ChatGPT
by Der_Einzige on Friday March 10, 2023 @10:10AM
(#35095258)
Depends on the LLM. Character based LLMs exist, and even have advantages vs regular LLMs...
I was having some gourmet crème brûlée with my friend Zoë at the café near the entrepôt in Åland's capital city and that made me realize it's but your naïveté when you say English is contained in mere 26 letters, for there is a soupçon of über important words that have diacritical marks in them.
There’s something interesting in the fact that an image based system doesn’t need as much complexity to capture a semantic model as a verbal system does; I think there’s maybe a parallel there to the way that human minds find it easier to just ‘visualize’ some things as a basis for reasoning about them, but if we can’t ‘visualize’ and instead have to ‘think things through’ it’s a more intensive process.
Like, GPT has well known trouble counting - ask it for five things and it will give you four or six. Humans can offload some thinking about counting to visual/spatial reasoning though.