News for hackers. Stuff that matters.

A comment on Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LLM, TTS, auth, payments). Parent comment

Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by kjok on Friday August 04, 2023 @05:39PM (#37006254)
What's the reason for the high inference latency? Any ideas on how this could be improved?
[ Parent | on HN ]
Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by drewbitt on Saturday August 05, 2023 @12:29AM (#37009033)
It has been sped up, but still not fast enough for this use case. https://github.com/manmay-nakhashi/tortoise-tts-fastest
[ Parent | on HN ]
Re:Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LL... by nielsinho on Saturday August 05, 2023 @05:31AM (#37010422)
TorToiSe is composed of many large models: GPT-2 for text encodings, as well as a large VQVAE encoder + large diffusion model decoder.

Only the big spaghetti inference code (+ weights) has been published, so there's a high entrance barrier for re-training / improving it.

[ Parent | on HN ]