News for hackers. Stuff that matters.

A comment on WhisperFusion: Ultra-low latency conversations with an AI chatbot.

Re:WhisperFusion: Ultra-low latency conversations with an AI... by jpcl on Thursday January 25, 2024 @10:19AM (#39130421)
Hi, I used the [WhisperSpeech](https://github.com/collabora/WhisperSpeech) model for the TTS part after I did some serious torch.compile optimizations to bring the latency down. The Whisper speech recognition and the LLM were optimized through TensorRT-LLM by Marcus and Vineet.

It's not perfect but I am still extremely proud of how it came out. :)

[ Parent | on HN ]
Re:WhisperFusion: Ultra-low latency conversations with an AI... by renus on Thursday January 25, 2024 @10:54AM (#39130863)
WhisperFusion is fully open-source - https://github.com/collabora/WhisperFusion
[ Parent | on HN ]
Re:WhisperFusion: Ultra-low latency conversations with an AI... by stiffler01 on Thursday January 25, 2024 @11:01AM (#39130948)
Tried this on 4090 and the responsiveness and real-time communication it offers are truly impressive. It has significantly improved my overall experience, especially in scenarios where minimal delay is crucial.

Compared to WhisperFusion, Rabbit R1 feels like it's stuck in the past, they could maybe use the OpenSource WhisperFusion.

[ Parent | on HN ]