News for hackers. Stuff that matters.

A comment on Bark – GPT-style realistic multilingual text-to-speech generation. Parent comment

Re:Bark – GPT-style realistic multilingual text-to-speech ge... by jamilton on Thursday April 20, 2023 @02:35PM (#35644242)
From the readme: >Bark has the capability to fully clone voices - including tone, pitch, emotion and prosody. The model also attempts to preserve music, ambient noise, etc. from input audio. However, to mitigate misuse of this technology, we limit the audio history prompts to a limited set of Suno-provided, fully synthetic options to choose from for each language.

It's not immediately clear how the audio history prompts are created.

[ Parent | on HN ]
Re:Bark – GPT-style realistic multilingual text-to-speech ge... by joshjob42 on Thursday April 20, 2023 @03:14PM (#35644798)
I don't know how they're made exactly, but one can just edit the code a bit and delete the restriction to just the given audio history prompts. It's literally just enforced, affect, with a simple "assert" command.
[ Parent | on HN ]
Re:Bark – GPT-style realistic multilingual text-to-speech ge... by gkucsko on Thursday April 20, 2023 @03:19PM (#35644854)
history prompts are just unconditionally generated TTS from the same model. any of those can be used as history, but for convenience 10 are provided for each language (to generate things with consistent voices)
[ Parent | on HN ]
Re:Bark – GPT-style realistic multilingual text-to-speech ge... by turnsout on Thursday April 20, 2023 @03:36PM (#35645111)
So the history prompts are collections of text/audio pairs?
[ Parent | on HN ]
Re:Bark – GPT-style realistic multilingual text-to-speech ge... by gkucsko on Thursday April 20, 2023 @03:56PM (#35645414)
history is semantic, coarse and fine. so essentially the same thing thats getting generated just using it as an input before the generation
[ Parent | on HN ]
Re:Bark – GPT-style realistic multilingual text-to-speech ge... by CamperBob2 on Saturday April 22, 2023 @02:18AM (#35663897)
So how do you clone an existing speaker's voice? That's the part I don't get.
[ Parent | on HN ]