News for hackers. Stuff that matters.

A comment on Show HN: A real time AI video agent with under 1 second of latency. Parent comment

Re:Show HN: A real time AI video agent with under 1 second o... by shtack on Tuesday October 01, 2024 @07:12PM (#41715345)
The model doing the heavy lifting is https://github.com/Rudrabha/Wav2Lip

Mic permissions on mobile are tricky, which might have been your issue? Note in this prototype you also need to hold the blue button down to speak.

[ Parent | on HN ]
Re:Show HN: A real time AI video agent with under 1 second o... by leobg on Wednesday October 02, 2024 @03:50AM (#41718215)
Interesting. I didn’t think you could get anything close to realtime with Wav2Lip.
[ Parent | on HN ]
Re:Show HN: A real time AI video agent with under 1 second o... by shtack on Wednesday October 02, 2024 @08:16AM (#41719766)
With a dedicated GPU and some cleverness it can be relatively quick. I split the response on punctuation and generate smaller clips in a pipeline. I haven't taken the model apart to try streaming the frames coming out of ffmpeg yet, but that would probably help a lot.
[ Parent | on HN ]