As for the Mandarin tones, the model might have mixed it up with the tones from a dialect like Cantonese. It’s interesting to discover how much difference a more specific prompt could make.

I don't know if my iOS app is using GPT-4o, but asking it to translate to Cantonese gives you gibberish. It gave me the correct characters, but the Jyutping was completely unrelated. Funny thing is that the model pronounced the incorrect Jyutping plus said the numbers (for the tones) out loud.

Not that different at all.

If you read into the details on openai's site, a lot of this stuff is clearly marked as english-first. For some written languages...noted as anything using non-roman characters, so most of asia, it basically doesn't work.

This really isn't surprising. Look at Google Home and Alexa. When they first came out, if you weren't a white male from the west coast... the accuracy of translating commands dropped DRAMATICALLY. Because it was programmed, designed and tested by majority of white tech bros in SF/Seattle. They've gotten a lot better over the last 5+ years. But I think you'll see OpenAI take this route.

But that's ok. They have to start somewhere. Once they get the model working really well for one language they can expand into similar ones with relatively little work. The more different the language, the more hard work and "local input" (ex: natives of the language) will be required for adaptation. But the basic text translations are still already way better than they used to be.

An interesting point, I tend to have better outcomes by using my heavily accented ESL English, than my native pronunciation of my mother tongue I'm guessing it's part of the tech work force being a bit more multicultural than initially thought, or it just being easier to test with

It's a shame, because that means I can use stuff that I can't recommend to people around me

Multilingual UX is an interesting painpoint, I had to change the language of my account to English so I could use some early Bard version, even though It was perfectly able to understand and answer in Spanish

You also get the synchronicity / four minute mile effect egging on other people to excel with specialized models, like Falcon or Qwen did in the wake of the original ChatGPT/Llama excitement.

Racist post

What? Did it seriously work worse for women? Spurce?

(accents sure)

In my experience, when someone says a project was programmed by "white men from the west coast", it was actually made by Chinese or Indian immigrants.

(Siri's original speech recognition was a combination of Swiss-Germans and people from Boston.)

And it certainly wouldn't be tested by them either way. Companies know how to hire QA contractors.

People always say tech workers are all white guys -- it's such a bizarre delusion, because if you've ever actually seen software engineers at most companies, a majority of them are not white. Not to mention that product/project managers, designers, and QA are all intimately involved in these projects, and in my experience those departments tend to have a much higher ratio of women.

Even beside that though -- it's patently ridiculous to suggest that these devices would perform worse with an Asian man who speaks fluent English and was born in California. Or a white woman from the Bay Area. Or a white man from Massachusetts.

You kind of have a point about tech being the product of the culture in which it was produced, but the needless exaggerated references to gender and race undermine it.

It was good of them to put in example failures.

I think there is too much focus on tones in beginning Chinese. Yes, you should get them right, but no, you'll get better as long as you speak more, even if your tones are wrong at first. So rather than remember how to say fewer words with the right tones, you'll get farther if you can say more words with whatever tones you feel like applying. That "feeling" will just get better over time. Until then, you'll talk as good as a farmer coming in from the country side whose first language isn't mandarin.

I couldn’t disagree more. Everyone can understand some common tourist phrases without tones - and you will probably get a lot of positive feedback from Chinese people. It’s common to view a foreigner making an attempt at Mandarin (even a bad one) as a sign of respect.

But for conversation, you can’t speak Mandarin without using proper tones because you simply won’t be understood.

That really isn't true, or at least it isn't true with some practice. You don't have to consciously think about or learn tones, but you will eventually pick them anyways (tones are learned unconsciously via lots of practice trying to speak and be understood).

You can be perfectly understood if you don't speak broadcast Chinese. There are plenty of heavy accents to deal with anyways. Like Beijing 儿化 or the inability of southerners to pronounce sh very differently from s.