News for hackers. Stuff that matters.
Re:OpenAI chief goes before US Congress to propose licenses ... by chaxor on Tuesday May 16, 2023 @10:58AM (#35962771)
Of course it predict the next token. Every single person on earth knows that so it's not worth repeating at all.

As for the fact that it gets things wrong sometimes - sure, this doesn't say it actually learned every algorithm (in whichever model you may be thinking about). But the nice thing is that we now have this proof via category theory, and it allows us to both frame and understand what has occurred, and to consider how to align the systems to learn algorithms better.

Re:OpenAI chief goes before US Congress to propose licenses ... by rdedev on Tuesday May 16, 2023 @11:09AM (#35962903)
The fact that it sometimes fails simple algorithms for large numbers but shows good performance in other complex algorithms with simple inputs seems to me that something on a fundamental level is still insufficient
Re:OpenAI chief goes before US Congress to propose licenses ... by zamnos on Tuesday May 16, 2023 @11:34AM (#35963265)
Insufficient for what? Humans regularly fail simple algorithms for small numbers, nevermind large numbers and complex algorithms
Re:OpenAI chief goes before US Congress to propose licenses ... by starlust2 on Tuesday May 16, 2023 @12:23PM (#35964076)
You're focusing too much on what the LLM can handle internally. No LLMs aren't good at math, but they understand mathematic concepts and can use a program or tool to perform calculations.

Your argument is the equivalent of saying humans can't do math because they rely on calculators.

In the end what matters is whether the problem is solved, not how it is solved.

(assuming that the how has reasonable costs)

Re:OpenAI chief goes before US Congress to propose licenses ... by ipaddr on Tuesday May 16, 2023 @03:19PM (#35966389)
Humans are calculators
Re:OpenAI chief goes before US Congress to propose licenses ... by glitcher on Tuesday May 16, 2023 @11:14AM (#35962978)
> Of course it predict the next token. Every single person on earth knows that so it's not worth repeating at all

What's a token?

Re:OpenAI chief goes before US Congress to propose licenses ... by visarga on Tuesday May 16, 2023 @11:46AM (#35963470)
A token is either a common word or a common enough word fragment. Rare words are expressed as multiple tokens, while frequent words as a single token. They form a vocabulary of 50k up to 250k. It is possible to write any word or text in a combination of tokens. In the worst case 1 token can be 1 char, say, when encoding a random sequence.

Tokens exist because transformers don't work on bytes or words. This is because it would be too slow (bytes), the vocabulary too large (words), and some words would appear too rarely or never. The token system allows a small set of symbols to encode any input. On average you can approximate 1 token = 1 word, or 1 token = 4 chars.

So tokens are the data type of input and output, and the unit of measure for billing and context size for LLMs.