News for hackers. Stuff that matters.
Re:ask HN: How does an LLM like GPT3 perform addition/division? by unlikelymordant on Thursday March 16, 2023 @07:41AM (#35180689)
at present, my understanding is that it is purely linguistic. If you ask "what do you call the wife of a king", it will answer "queen" because that is the highest likelyhood next token. If you ask "what is 2 + 3", it will answer "5" because that is the highest likelyhood next token purely from a linguistic point of view, i.e. no math has been performed.

All the emergent properties of chatgpt being so good, is that it is very good at predicting word sequences that, when evaluated by a human, sound like it is doing all sorts of higher level logic.

Re:ask HN: How does an LLM like GPT3 perform addition/division? by diatone on Thursday March 16, 2023 @09:24AM (#35181608)
Would be fun to see a breakdown of how some arithmetic prompts are tokenised, to help understand this. eg: is the number “1.23e+5” one token?