News for hackers. Stuff that matters.
Re:Translating Akkadian clay tablets with ChatGPT? by H8crilA on Tuesday May 16, 2023 @04:44AM (#35959145)
I am quite sure GPT can undo tokenization if needed. It has no problem handling misspellings, plus you can just ask it to "spell things" L-I-K-E S-O.
Re:Translating Akkadian clay tablets with ChatGPT? by RugnirViking on Tuesday May 16, 2023 @08:10AM (#35960709)
I'm pretty sure there is a spellchecker ran before the tokenisation

also it is often wrong with spelling stuff out like that, especially with rarer or more interesting tokens