News for hackers. Stuff that matters.
Re:Toki Pona: an attempted universal language with only ~120... by coder543 on Sunday August 13, 2023 @04:50PM (#37114310)
In this case, I validated against the hand-translated example in the main link. It did fine.

This is probably easy mode for it, given the tiny vocabulary at hand, even though it has surely seen very little Toki Pona in the wild. Large language models are extremely good at translation. I’ve personally tested ChatGPT-4 against Google Translate and other translators on some hard translations (languages that are not similar to English, such as Chinese), and it has consistently provided better translations than Google Translate in my opinion. For validation purposes, news stories that have been reported worldwide make things easier, since you can establish the facts of the translation by hand, and then compare what comes out.

What does your comment actually add to the discussion? Yes, ChatGPT can be wrong. So can Google Translate!

If you’d like to provide a clear example of ChatGPT-4 (not ChatGPT-3.5) making up a Toki Pona translation that is completely disconnected from reality, please do. That would be legitimately interesting conversational material. General cynicism isn’t.

ChatGPT absolutely has limitations, and translation is a hard problem, even for the industry-standard tools. Crossvalidation with multiple translation tools and ideally humans is important for any translations that actually matter.

Re:Toki Pona: an attempted universal language with only ~120... by NoZebra120vClip on Sunday August 13, 2023 @04:54PM (#37114356)
Since LLMs' only valid uses involve entertainment and creativity, and Toki Pona is more or less a recreational language without serious application, I think they go together like two peas in a pod.
Re:Toki Pona: an attempted universal language with only ~120... by MostlyStable on Sunday August 13, 2023 @05:09PM (#37114521)
>only valid use...

Well. That's an opinion I suppose.

-edit- After posting, I decided this was too snarky/low value on it's own. So to add:

I personally get quite a bit of value out of it as an ad-hoc German tutor and as one-among-many tools for research and learning on new topics for which I am not already an expert.

I've also used it to help me make and troubleshoot very simple code in languages that I am not an expert in.

I personally have trivially found lots of uses that probably all cluster around "educational" to one degree or another.

Is it perfect? Absolutely not. Does it hallucinate/get things wrong? Absolutely. Is it useful despite those limitations? Emphatically yes.

This idea that its "only valid use" is entertainment doesn't even make sense to me. "Entertainment" is perhaps where I find it the most deficient relative to other current options.

Re:Toki Pona: an attempted universal language with only ~120... by dotancohen on Sunday August 13, 2023 @05:48PM (#37114873)

  > In this case, I validated against the hand-translated example in the main link. It did fine.
Presumably that hand-translated example was also part of the training data.
Re:Toki Pona: an attempted universal language with only ~120... by coder543 on Sunday August 13, 2023 @05:52PM (#37114910)
Not likely. The article says it was posted 1 year ago. ChatGPT’s training data cutoff is September 2021, so no, that would not make sense.
Re:Toki Pona: an attempted universal language with only ~120... by mistercow on Sunday August 13, 2023 @09:02PM (#37116280)
I've done a fair amount of translating Toki Pona based on my own sentences, and based on recent online posts. None of that would have been in the training set. It does a good job. Since Toki Pona translation is "lossy", you can even prompt it to give you multiple interpretations and discussion.
Re:Toki Pona: an attempted universal language with only ~120... by janAkali on Sunday August 13, 2023 @10:56PM (#37116951)
Another challenge, is that toki pona requires a lot of context, not only of previous sentences, but visual and communal. For example, I can say 'soweli lili' and point to a cat, then in all further conversations 'soweli lili' will mean exactly a 'cat' until specified.

I could use sentence that could mean literally hundred thousand of different things, but if I explain it properly once, you are expected to keep it as context.

ChatGPT4 will struggle to keep all context as it will surely accumulate.