Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by modeless on Sunday November 19, 2023 @08:50PM
(#38341138)
Yes, this is a good idea. Too many good ideas, not enough time!
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by woodson on Monday November 20, 2023 @12:06AM
(#38342508)
A simple correlation of audio chunks from microphone and from the TTS should be enough to tell which parts in the input stream are re-recorded TTS. Much simpler, no?
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by modeless on Monday November 20, 2023 @12:18AM
(#38342631)
It's not so simple when the impulse response of the room and mic and speakers are all unknown, possibly changing, plus unknown background sounds as well, possibly at a very high level. and there's also unknown latency which can be quite large especially in the networked case, and maybe some codecs, and maybe some audio "enhancement" software the OEM installed on the user's machine. Also, ideally the computer would be able to hear the user even while it is speaking.
Echo cancellation is non-trivial for sure.
Re:StyleTTS2 – open-source Eleven Labs quality Text To Speech
by IanCal on Monday November 20, 2023 @06:21AM
(#38346406)
Could it be done more reasonably with the transcription? With diarisation the logic of "someone is saying exactly / almost exactly what I'm saying, that's probably me" might be pretty reasonable.
Maybe you can rely on some sort of speaker identification to sort this out?
https://github.com/openai/whisper/discussions/264