CROON_tv

What I'd want to see next to accuracy is tail latency. In a real-time use, deciding when a spoken sentence is finished, a general LLM with the same prompt was slower and more hesitant for us than Jev, even though both cost about the same.

Similar result in production for me. I compared Jev with a general LLM (DeepSeek) as the decider for when to cut live speech into sentences, on the same 226 segments. With Jev the longest held segment was 17 s. With DeepSeek it was 40 s.

I use Jev in an iPhone app that dubs whatever the phone plays in real time. It decides whether a translated sentence is complete enough to speak now or should wait.

It's fast and cheap: about 0.35 s per decision, about 2 cents per hour of audio.

Thanks! Translation is done with ASR, an LLM and TTS.

The delay is about 4 seconds on average. It could be faster, but translating half a sentence gets confusing, so it waits for the sentence to end.

The LLM handles idioms.

I'm Japanese, and most of the tech podcasts I want to hear are only in English. I wanted to just listen, not read subtitles.

So I made an iPhone app that dubs whatever your phone is playing into your language as it plays, in a voice like the speaker's. It works on top of any app.

Launch video: https://x.com/CROON_tv/status/2102377490367046122

The link is a TestFlight beta, and it needs the iOS 27.2 beta for now.

Would you use this?