p.s. Seems every day on here I'm reminding people from the US that their country is not the world, their version of English is spoken and spelt in just one country, etc etc etc. Maybe I should give up doing that.
The most classic maybe was a linguist (yes, amazing) who on her blog was gloating that her variety of US English was particularly rich, in having almost every vowel sound in English. I looked at her list.. it didn't include the 'o' sound in 'pot'..
I agree that the mouth shapes are garbage in this demo. A shame because that is the most interesting & important part of animating speech.
I'm happy that this article has triggered some people curiosity and can maybe evolve in something more advanced
*(just another rabbit hole actually, Unicode got plenty of them already)
"text engine developers hate him"
It definitely is! The Latin alphabet gets a special place at the start of the range, in order to be compatible with ASCII, an English-language character set.
I didn't criticise it, did I? I just pointed it out. So I don't need to see any better alternative.
But I don't follow your argument - it's not English centric because they were only bending over backwards to make sure that the English-speaking world accepted it by designing it around them?
The only sounds I would be very confident at recovering sounds from images for ‘th’, ‘w’, ‘o’, & indistinguishable voiced/unvoiced pairs like ‘b’/‘p’ or ‘v’/‘f’. Basically, the sounds a ventriloquist has to cheat.
Maybe that's why most animation studios don't bother.
Dubs. 90% of your viewers will see it in a foreign language, dubbed, where all your lipsyncing effort is useless anyway.
In fact, lipsyncing ut to english makes it look worse in the dub than having random nonsense movements.
I remember distinctly this wonderful line from the 5th element:
- Are you classified as human?
- Negative. I'm a meat Popsicle!
Which they translated to french into:
- ...?
- Negatif. Je suis une mite en pull over !
Which has not relation at all to the initial line, and literally translateds mean "I'm a moth wearing a sweater".
But it's equally nonsensical, definitely as funny, and the beautiful thing is, the lipsync is near perfect.
However it's a lot of work, requires very talented people to achieve, and is mostly useless in the eyes of the right owners: money will come anyway if the movie is average, so why bother making it better ?
As child (due to deafness until I was almost 5) I relied a lot on lip reading, and it was very noticeable if something was dubbed.
Thanks for this extremely intriguing idea to use emoji to depict a talking person visually.
You know, some animated films get it right. Sometimes I clearly see utterances like «Thank you!», «Okay», but more often not. Especially older animations don't care and just let the character move the mouth in very simplistic ways: «Babbabbabaa Baababba ba baaa.»
Similarly for that emoji. It is moving the mouth in quite arbitrary ways. It looks like: «Sobbabbabee be <grin> seebee <grin> <frown> babbaa».
To solve this problem we need about twelve emoji for utterances: «ah», «ay», «ee», «oh», «oo», «b», «d», «f», «l», «n», «r» and «s». The «r» emoji must be animated (so we see the trilling tongue). The other sounds are either invisible like «k» or cover several different sounds like «b» also covering «m» and «p».
«Okay» would be rendered by: «oh», «oo», «oo» (standing for «k» wich is invisible) «ay» and «ee».
Edit: Add «th». This is an extremely simple sound to lipread. I remember my delight when I learnt English in my youth and in an movie suddenly realised that an actor said «Thank you».