Show HN: I built a sub-500ms latency voice agent from scratch

Re:Show HN: I built a sub-500ms latency voice agent from scr... by nicktikhonov on Monday March 02, 2026 @05:58PM (#47225434)

Very cool! starred and on my reading list. Would love to chat and share notes, if you'd like

[ Parent | on HN ]

Re:Show HN: I built a sub-500ms latency voice agent from scr... by alfalfasprout on Monday March 02, 2026 @07:39PM (#47226365)

Also consider using Cerebras' inference APIs. They released a voice demo a while back and the latency of their model inference is insane.

[ Parent | on HN ]

Re:Show HN: I built a sub-500ms latency voice agent from scr... by ilaksh on Tuesday March 03, 2026 @02:06AM (#47229121)

I tried to use Cerebras and it was unbeatable at first, but the client didn't want to pay $1300 a month and the $50/month or pay as you go was just not reliable. It would give service unavailable errors or falsely claim we were over our rate limit.

Also Groq is very fast, but the latency wasn't always consistent and I saw some very strange responses on a few calls that I had to attribute to quantization.

[ Parent | on HN ]

Re:Show HN: I built a sub-500ms latency voice agent from scr... by riquito on Tuesday March 03, 2026 @01:05AM (#47228716)

You may be interested in gemini-2.5-flash-preview-tts

Text in, audio out, so you can merge in a single step LLM+TTS (streamable)

https://ai.google.dev/gemini-api/docs/models/gemini-2.5-flas...

[ Parent | on HN ]