News for hackers. Stuff that matters.

A comment on Observation: I asked ChatGPT to notify me when our chat exceeds 200 words. Parent comment

Re:Observation: I asked ChatGPT to notify me when our chat e... by koheripbal on Saturday April 08, 2023 @09:43PM (#35499356)
I honestly don't understand why they don't just set words as tokens. Is the dictionary really that big?
[ Parent | on HN ]
Re:Observation: I asked ChatGPT to notify me when our chat e... by cyclotron3k on Saturday April 08, 2023 @10:27PM (#35499569)
Yes, it really is! Because you're not actually dealing with a dictionary of English words - you're dealing with all possible combinations of characters, from all unicode character sets.

Here's a really good explanation: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

[ Parent | on HN ]
Re:Observation: I asked ChatGPT to notify me when our chat e... by qup on Saturday April 08, 2023 @10:25PM (#35499560)
Your question belies the truth in your statement.
[ Parent | on HN ]