News for hackers. Stuff that matters.
Re:LLM Powered Autonomous Agents by zwaps on Tuesday June 27, 2023 @03:12AM (#36489813)
Basically the latter
Re:LLM Powered Autonomous Agents by weinzierl on Tuesday June 27, 2023 @03:55AM (#36490103)
The tokenization algorithms I encountered all had around 50000 tokens, which fits nicely into (and makes good use of) a 16-bit number. Is this just a coincidence or does it have advantages for the token to be a 16-bit representable number?
Re:LLM Powered Autonomous Agents by danuker on Tuesday June 27, 2023 @05:32AM (#36490758)
I suspect it being 16 bit instead of 32 bit means more of them can get packed more tightly. Some instructions can operate on them in parallel.

But I personally think it's a coincidence, and it just so happens that 50k tokens are enough for the level of complexity the models have right now.

Re:LLM Powered Autonomous Agents by sebzim4500 on Tuesday June 27, 2023 @06:12AM (#36490991)
Probably a coincidence. The GPT-4 and GPT-3.5 tokenizer has 100k tokens.