Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on LLMs use a surprisingly simple mechanism to retrieve some stored knowledge. Parent comment
If the model in question has a token embedding length of 1024, even if it was a 1 bit quantization, each token has 2^1024 possible values.
If the context length is 32,000 tokens, there are 32,000^2^1024 possible inputs.
There are no comments on this comment yet.
If the model in question has a token embedding length of 1024, even if it was a 1 bit quantization, each token has 2^1024 possible values.
If the context length is 32,000 tokens, there are 32,000^2^1024 possible inputs.