News for hackers. Stuff that matters.

A comment on Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LLM, TTS, auth, payments).

Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by benreesman on Friday August 04, 2023 @04:54PM (#37005752)
Related, what’s the current SOTA on STT models freely available? T5 is pretty good but the closed Google and Meta stuff seems better.
[ Parent | on HN ]
Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by nielsinho on Friday August 04, 2023 @05:05PM (#37005883)
TorToiSe (https://github.com/neonbjb/tortoise-tts) produces the best quality speech of any freely available model. However, its long inference times makes it impractical for voice chatbots like Gdansk.
[ Parent | on HN ]
Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by kjok on Friday August 04, 2023 @05:39PM (#37006254)
What's the reason for the high inference latency? Any ideas on how this could be improved?
[ Parent | on HN ]
Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by drewbitt on Saturday August 05, 2023 @12:29AM (#37009033)
It has been sped up, but still not fast enough for this use case. https://github.com/manmay-nakhashi/tortoise-tts-fastest
[ Parent | on HN ]
Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by nielsinho on Saturday August 05, 2023 @05:31AM (#37010422)
TorToiSe is composed of many large models: GPT-2 for text encodings, as well as a large VQVAE encoder + large diffusion model decoder.

Only the big spaghetti inference code (+ weights) has been published, so there's a high entrance barrier for re-training / improving it.

[ Parent | on HN ]