Show HN: WhisperFusion – Ultra-low latency conversations with an AI chatbot

Re:Show HN: WhisperFusion – Ultra-low latency conversations ... by pksebben on Monday January 29, 2024 @01:07PM (#39179812)

I wrote a sort of toy version of this a little while ago using Vosk and a variety of TTS engines, and the solution that worked mostly-well was to have a buffer that waited for audio that filled until a pause of so many seconds, then it sent that to the LLM.

With the implementation of tools for GPT, I could see a way to having the model check if it thinks it received a complete thought, and if it didn't, send back a signal to keep appending to the buffer until the next long pause. The addition of a longer "pregnant pause" timeout could have the model check in to see if you're done talking or whatever.

[ Parent | on HN ]

Re:Show HN: WhisperFusion – Ultra-low latency conversations ... by renus on Monday January 29, 2024 @01:38PM (#39180367)

To streamline the experience we don't send the transcription to the LLM after the pause, since we are using the time we wait for the end of sentence trigger (pause) to generate the LLM and text-to-speech output. So ideally once we detected the pause, we already processed everything.

[ Parent | on HN ]