News for hackers. Stuff that matters.
Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by rjh29 on Monday March 27, 2023 @11:38PM (#35335583)
I have tested it :) I've asked it to translate English fictional text into Japanese, it falls over completely. It's unnatural and often makes no sense at all. It doesn't compare whatsoever to a typical professional translation (which are often not that idiomatic either), let alone a really good one.

I'm sure it'll be doing that in five years, but not now.

One interesting thing is that's it's nondeterministic, so sometimes 'For chrissakes' turns to ちくしょう (Damn!) but sometimes to クリスのために (for Chris' sake). Sometimes 'the goddamn door' turns into クソドア ('shit door', unnatural), sometimes the goddamn changes the phrasing of the whole sentence instead. If you run it five times and take the best sentences out of all five runs it's probably quite good.

Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by og_kalu on Monday March 27, 2023 @11:42PM (#35335613)
Mind sharing output ?

I mean i can if you want (chinese though) but enough people lie on the internet.

Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by bumbledraven on Monday March 27, 2023 @11:39PM (#35335590)
Would you mind sharing a sample input and output that you found unsatisfactory?
Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by rjh29 on Monday March 27, 2023 @11:52PM (#35335691)
I don't think we disagree. The video says the translation will be "readable" but needs several days of an experienced editor passing over it. That's an amazing result, but again, it's not as good as a human yet. It's way faster and it'll make media accessible to tons of people.

Like he says, there's lots of ambiguity in Japanese that needs to be handled, gender not being specified until later, etc. and an editor would need to spend time going over it - but it saves months of traditional work. There are words and _concepts_ that are hard to translate, there are cultural issues, dialects, slang, registers. So yeah it'll make the media accessible, but it won't be as a good as a skilled translator.

Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by bumbledraven on Tuesday March 28, 2023 @12:05AM (#35335787)
He didn't merely say "readable"; he said (as I quoted in GP) "the basic quality of the translation is as good as a human might do."
Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by raincole on Monday March 27, 2023 @11:57PM (#35335732)
Last night I used GPT-4 to translate the first several pages of Ted Chiang's Lifecycle of Software Objects (a sci-fi piece) from English to Chinese. I'd say it's about as good as me, save a few minor errors. It's safe to say it performs better than a "tired me", and some translators I've seen on the market.

I'm a native speaker of Chinese, but not a professional translator.

Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by lowefk on Tuesday March 28, 2023 @01:25AM (#35336277)
Here is GPT-4's translation, and I find no issues: https://imgur.com/a/oOtf4RD
Re:ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks by kolinko on Tuesday March 28, 2023 @01:46AM (#35336424)
It may depend on a language. For Polish - which is considered one of the most difficult languages due to various forms of words, it works almost perfect - on par with average human translators.