WhisperFusion: Ultra-low latency conversations with an AI chatbot

Posted by mfilion on Thursday January 25, 2024 @09:52AM
from the collabora.com dept.

https://www.collabora.com/news-and-blog/news-and-events/whisperfusion-ultra-low-latency-conversations-with-an-ai-chatbot.html

( 4 comments | 8 points | on HN )

Re:WhisperFusion: Ultra-low latency conversations with an AI... by jpcl on Thursday January 25, 2024 @10:19AM (#39130421)

Hi, I used the [WhisperSpeech](https://github.com/collabora/WhisperSpeech) model for the TTS part after I did some serious torch.compile optimizations to bring the latency down. The Whisper speech recognition and the LLM were optimized through TensorRT-LLM by Marcus and Vineet.

It's not perfect but I am still extremely proud of how it came out. :)

[ Parent | on HN ]

Re:WhisperFusion: Ultra-low latency conversations with an AI... by renus on Thursday January 25, 2024 @10:54AM (#39130863)

WhisperFusion is fully open-source - https://github.com/collabora/WhisperFusion

[ Parent | on HN ]

Re:WhisperFusion: Ultra-low latency conversations with an AI... by stiffler01 on Thursday January 25, 2024 @11:01AM (#39130948)

Tried this on 4090 and the responsiveness and real-time communication it offers are truly impressive. It has significantly improved my overall experience, especially in scenarios where minimal delay is crucial.

Compared to WhisperFusion, Rabbit R1 feels like it's stuck in the past, they could maybe use the OpenSource WhisperFusion.

[ Parent | on HN ]

Re:WhisperFusion: Ultra-low latency conversations with an AI... by unraveller on Friday January 26, 2024 @03:07AM (#39139972)

I wish this was available in perplexity labs, I wouldn't even use it for long things or waffling, just for those commands I use over and over again (like "do it again but for X without Y") and know exactly how to say because I'm sick of typing them.

[ Parent | on HN ]