News for hackers. Stuff that matters.
Re:Language models cost 10x more in some languages than others by codetrotter on Thursday May 18, 2023 @03:11AM (#35984916)
https://medium.com/do-you-not-like-language/is-mandarin-more...

It’s possible Mandarin has even higher efficiency. And there are very many people who speak Mandarin. You are expecting English to come out on top. But do not be surprised if in the end it will be Mandarin.

Re:Language models cost 10x more in some languages than others by shjake on Thursday May 18, 2023 @03:17AM (#35984948)
It’s not like one has too fully dominated the other.

I can’t imagine mandarin ever becoming widespread in the west unless there are some fundamental changes to Chinese society and culture.

Similarly western tech companies have huge issues in penetrating the Chinese market.

Re:Language models cost 10x more in some languages than others by carlmr on Thursday May 18, 2023 @03:25AM (#35984993)
I think if Mandatin started using pinyin or it could start dominating. The language itself isn't that much more difficult than other languages. You can get used to tones, too. The grammar is actually simpler than in English.

But the sheer number of characters to rote memorize makes it way too inefficient to learn for foreigners, and arguably even for natives, that I don't see it becoming a world language.

There's always the argument that pinyin isn't enough, that the characters are essentially useful. Which I think Vietnamese, which used to be based on the same writing system and is also tonal, proves wrong.

If you can't communicate with pinyin you can't communicate with the language, since it almost perfectly models the sounds.

Re:Language models cost 10x more in some languages than others by seanmcdirmid on Thursday May 18, 2023 @03:32AM (#35985024)
Chinese characters are used phonetically for a lot of minority (non-Chinese) languages in China. Which I don’t really get, but there it is. I guess there is legacy to consider, as the use of Chinese in writing systems predate pinyin, and there is no pressure to change.
Re:Language models cost 10x more in some languages than others by adelie on Thursday May 18, 2023 @04:03AM (#35985189)
> If you can't communicate with pinyin you can't communicate with the language, since it almost perfectly models the sounds.

hm, i'd say it's a bit more complicated than that. written and literary chinese has a different (often more concise) style than conversational chinese because certain things that would be ambiguous spoken aren't that way when written. similarly, japanese has a perfectly functional syllabary (hiragana), but people will use kanji anyways because it's a lot easier to parse at a glance.

Re:Language models cost 10x more in some languages than others by nindalf on Thursday May 18, 2023 @03:46AM (#35985101)
It’s hard to predict future cultural relevance. No one in 1998 was predicting then global popularity of Korean film/TV/music within 2 decades. And yet it happened.

Chinese TV shows could become equally popular. Some will say you need to have freedom to create art but I’m pretty sure that the Mario movie or Fast X (two movies at the top at the moment) needed minimal artistic freedom. Fast X is actually a great example because that franchise goes out of its way to toe the CCP line.[1] It shows that it’s possible to make billions while also kowtowing to the Party.

[1] - John Cena apologises for calling Taiwan a country (https://www.nytimes.com/2021/05/25/world/asia/john-cena-taiw...)

Re:Language models cost 10x more in some languages than others by realusername on Thursday May 18, 2023 @03:52AM (#35985133)
> Chinese TV shows could become equally popular.

I personally do believe that you need enough artistic freedom to create those worldwide popular shows.

As an example, I'm watching Korean police shows which constantly have corruption as a main plot line, good luck filming corrupt cops in a series in China with the CCP.

What they can produce for now is limited to the dullest stories and that's not going to cut it.

Re:Language models cost 10x more in some languages than others by nindalf on Thursday May 18, 2023 @04:01AM (#35985176)
The most popular K-Dramas are the formulaic 1 guy and 1 girl, with a 3rd person for a love triangle. Don’t worry though, true love wins and the 3rd wheel finds someone as well. Eg: Hometown Cha-Cha-Cha.

CCP would heartily endorse these brainless plots. Consumers would eat it up.

Not everything needs to be Infernal Affairs. The vast majority of Korean TV isn’t Infernal Affairs. I will admit that China could never make Parasite, but that’s ok. The metric we’ve chosen is popularity, not Oscars.

Side note I’m a bit sad that Hong Kong can’t make Infernal Affairs type of movies anymore. It would show the Party in a bad light.

Re:Language models cost 10x more in some languages than others by realusername on Thursday May 18, 2023 @04:46AM (#35985453)
Oh yeah, I'm sure they will have their success as well, their drama series with the emperor in imperial China (whatever it's called, I don't know) is also very popular. It's just that relative to their massive size, they will stay underperforming as long as the CCP cuts every head which steps a little outside their comfort zone.
Re:Language models cost 10x more in some languages than others by ajuc on Thursday May 18, 2023 @04:18AM (#35985284)
> good luck filming corrupt cops in a series in China with the CCP

I don't know about China, but when Poland was communist and there was widespread censorship - it was still OK to talk about corruption, it was just supposed to be shown in a way that makes it clear that corruption is western influence and communism is getting rid of it.

Funnily enough - Polish cinema was probably better artistically during communism, because it had to work hard to work around the censorship. So you had movies that were trying to be universal, say the things that can't be said using symbols, etc.

Modern Polish cinema is pretty awful, mostly shitty romcoms.

I'm not saying censorship is good, it's evil obviously, but the effect on art can be counterintuitive.

Re:Language models cost 10x more in some languages than others by yorwba on Thursday May 18, 2023 @04:29AM (#35985350)
Having corrupt cops in a Chinese series is no problem at all, as long as they're the bad guys and are defeated by the heroes in the end, or at least have a redemption arc.

Here's a Zhihu question where someone asked to be recommended anti-corruption series, maybe you should watch some of the shows mentioned in the replies before concluding that only dull stories can be produced: https://www.zhihu.com/question/502243280

Re:Language models cost 10x more in some languages than others by shjake on Thursday May 18, 2023 @06:02AM (#35985978)
> Korean film/TV/music within 2 decades.

True. But it’s still rather limited and almost nobody is learning Korean just because of that.

e.g. in the 60s French cinema was pretty popular for some time in the US in a similar way. Barely anybody learnt French because of that (and it would’ve been way, way easier than Chinese or Korean)

Re:Language models cost 10x more in some languages than others by realusername on Thursday May 18, 2023 @03:49AM (#35985116)
> I can’t imagine mandarin ever becoming widespread in the west unless there are some fundamental changes to Chinese society and culture.

I could imagine it if a better pinyin similar as Vietnamese would become the most popular way to write mandarin in China. This would lift a lot of the difficulty of westerners.

This isn't going to happen though.

Re:Language models cost 10x more in some languages than others by hunglee2 on Thursday May 18, 2023 @03:14AM (#35984933)
not sure about Mandarin, I think it will follow a different path given that a significant degree of the meaning is implicit in Mandarin. LLM is trained in explicit (documented) information, so unclear in my mind right how whether this will be more or less efficient. Will read the link though - thanks!
Re:Language models cost 10x more in some languages than others by seanmcdirmid on Thursday May 18, 2023 @03:20AM (#35984965)
Given how Chinese talk about topics using code words to discuss controversial topics online (and avoid low effort keyword censorship), I wonder if an LLM could learn to do that given a censor in the training process?
Re:Language models cost 10x more in some languages than others by hunglee2 on Thursday May 18, 2023 @03:48AM (#35985112)
it's all code words, this is not as conspirational as imagined by non-language speakers. A great deal of normal conversation / interaction is idiomatic given that it is difficult to create new words in logographic languages. Your point is valid though - how does AI learn this? I don't think it can easily, it is akin for an AI to keep up with street slang, it couldn't guess it an advance it would have to be trained in it, which seems it will need greater human attendance than alphabetical languages!
Re:Language models cost 10x more in some languages than others by xgdgsc on Thursday May 18, 2023 @03:49AM (#35985120)
I used to set my OS and apps to English for easy search of function or bugs. But with the trend of design that emphasizes "flat" look, which often includes removing menu icons and keep only English text. I started switching back app languages to Chinese, where every menu text itself is already an "icon" that 's way easier to find on glimpse.
Re:Language models cost 10x more in some languages than others by fallingknife on Thursday May 18, 2023 @08:02AM (#35986755)
English is dominant, not because the number of speakers, but because they are so widespread. Mandarin has more, but they area almost all in China, whereas there is hardly a corner of the earth where you can't find an English speaker. So if you plan to travel to a lot of places, the most efficient language to learn is English. Between any two non-neighboring countries, the language with the highest number of mutual speakers is almost certainly English. I don't think any other language in history has had this level of network effect.