ChatGPT can now see, hear, and speak – openai.com

Re:ChatGPT can now see, hear, and speak – openai.com by dsp_person on Monday September 25, 2023 @02:24PM (#37648344)

Also curious to hear about your setup. Using whisper too? When I was experimenting with it there was still a lot of annoyance about hallucinations and I was hard coding some "if last phrase is 'thanks for watching', ignore last phrase"

I was just googling a bit to see what's out there now for whisper/llama combos and came across this: https://github.com/yacineMTB/talk

There's a demo linked on the github page that seems relatively fast at responding conversationally, but still maybe 1-2 seconds at times. Impressive it's entirely offline.

[ Parent | on HN ]

Re:ChatGPT can now see, hear, and speak – openai.com by modeless on Monday September 25, 2023 @07:06PM (#37651987)

Lol yeah the hallucinations are a huge problem. Likely solvable, I think there are probably some bugs in various whisper implementations that are making the problem worse than it should be. I haven't really dug in on that yet though. I was hoping I could switch to a different STT model more designed for real time like Meta's SeamlessM4T but it's still under a non-commercial license and I did have an idea that I might want to try making a product sometime. I did see that yacine made that version but I haven't tried it so I don't know how it compares to mine.

[ Parent | on HN ]

Re:ChatGPT can now see, hear, and speak – openai.com by QuantumG on Monday September 25, 2023 @11:01PM (#37653801)

Turn the volume on your microphone down and watch as Whisper just starts SCREAMING.

[ Parent | on HN ]