News for hackers. Stuff that matters.
Re:Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speec... by satvikpendem on Thursday March 05, 2026 @09:26AM (#47261862)
Why can't a speech to speech model do tool calls? Others like Gemini live do it just fine.
Re:Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speec... by d4rkp4ttern on Thursday March 05, 2026 @09:47AM (#47262113)
Ok, I was wrong. I just tested ChatGPT voice, Claude Voice and Gemini Live. And all three are able to do web search. For some reason, I thought when I tested ChatGPT voice a few weeks ago, it sometimes said it can’t directly open links, but it can do web search, which was strange.
Re:Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speec... by raw_anon_1111 on Thursday March 05, 2026 @09:54AM (#47262224)
If it is doing a tool call, it has to convert the speech to text or at least a JSON object of the necessary parameters for the tool and convert the result to speech doesn’t it? Is it truly speech to speech then?
Re:Nvidia PersonaPlex 7B on Apple Silicon: Full-Duplex Speec... by satvikpendem on Thursday March 05, 2026 @10:16AM (#47262522)
It's all tokens at the end of the day, not really text or video or audio, just like everything on a machine is just bits of 1s and 0s and it's up to the program to interpret them as a certain file format. These models are more speech-to-speech (+ text) in that they can recognize text tokens too. So the flow is, you ask it something, then,

Audio Tokens: "Let me check that for you..." (Sent to the speaker)

Special Token: [CALL_TOOL: get_weather]

Text Tokens: {"location": "Seattle, WA"}

Special Token: [STOP]

The orchestrator of the model catches the CALL_TOOL and then calls the tool, then injects this into the context of the audio model which then generates new tokens based on that.