News for hackers. Stuff that matters.
Re:Why is GPT-3 15.77x more expensive for certain languages? by biztos on Monday April 10, 2023 @03:49PM (#35516582)
Really, only Thai? Is there a reference for that? A quick search suggests it’s not the case, but I’m no expert.

As a lowly beginner I find the lack of word boundaries in Thai frustrating but I think it’s just that I have not yet learned to think in syllables, I’m still always sounding them out in my head until I have a word I recognize, there’s no flow.

This seems like something the LLMs should be very good at. Google Translate does OK-ish while Apple just throws up its hands in frustration and refuses to translate Thai texts.

Re:Why is GPT-3 15.77x more expensive for certain languages? by mgaunard on Tuesday April 11, 2023 @03:12AM (#35521844)
Read the Unicode standard, it covers all of these things.