ChatGPT can now see, hear, and speak – openai.com

Re:ChatGPT can now see, hear, and speak – openai.com by fragmede on Monday September 25, 2023 @09:35PM (#37653220)

Yeah they do. I was able to get ChatGPT-4 to transcribe 我哥哥高過他的哥哥, which says that they can. I did have to set the app to Chinese, and the original didn't work so I had to modify what I said slightly.

https://www.tiktok.com/t/ZT86psPxY/

Roughly translated, my older brother is taller than that other guy's older brother.

[ Parent | on HN ]

Re:ChatGPT can now see, hear, and speak – openai.com by modeless on Monday September 25, 2023 @10:18PM (#37653497)

Of course speech recognition works for Chinese. What it doesn't do is transcribe intonation and prosody in non-tonal languages. It's not even clear how one would transcribe such a thing as I'm not aware of a standard notation.

[ Parent | on HN ]

Re:ChatGPT can now see, hear, and speak – openai.com by fragmede on Friday September 29, 2023 @04:21AM (#37700786)

IPA format should cover that, no?

[ Parent | on HN ]

Re:ChatGPT can now see, hear, and speak – openai.com by modeless on Saturday September 30, 2023 @01:12AM (#37712928)

Maybe? I thought IPA was just phonetic but I see that it does have some optional prosody stuff that could in theory cover some of it. I'm not sure how standard or complete it really is in practice.

I haven't heard of any large datasets of IPA transcripts of speech with the detail necessary to train a fully realistic STT->LLM->TTS system. If you know of some that would be interesting to look at.

[ Parent | on HN ]