100% agree. I've been building MCPs that enable Claude (my agent of choice to have voice conversations with). These hook into my home maintenance system (https://homemaintlist.com) and workout tracking system (https://tallyup.codewalkers.net) plus several private mcp servers I've made to expose Claude to specific services. These all allow me to just have conversations with Claude and have all these very specific things available and take action on them.

Curious how you talk to Claude? I have some automations set up but I feel locked into Gemini on android if I want to be able to use a wakeword (e.g. while driving). I use Claude for non-voice but haven't found a convenient way to start up a chat with Claude.

Yeah...wish there was a better way than just opening the app and clicking the voice button. I haven't found one. My talking is usually in the car, so it's not too inconvenient for me to do that.

I see, yeah that works when I know I'll be in a situation for a while where I may want the option to say something or going over something like verbal brainstorming but I find I am much more often having a random thought and want to capture it right away without pulling out my phone or interrupting what I'm doing.

Can only hope google relaxes their limitations on choice of ai assistant. In the meantime I'm setting up a raspi with openwakeword to just act as an interface for voice to voice interactions for while I'm home.

I looked into this a bit yesterday and had similar issues. One hacky option on Mac could be to [1] use BlackHole virtual audio device as the system default input (which Claude expects), [2] vibe code some app that can pick your actual mic as the input device (browser engines seem to have a pretty good support for this stuff), [3] stab a speech-to-text model there (whisper or lighter) that detects the wake word from a stream and [4] proxies the audio after that to the output side of BlackHole. I did not try it myself yet, though.

[delayed]

Excellent points. We still need to employ good cybersecurity principles when granting AI access to tools and resources. Let’s not throw out everything learned in the last three decades. Keep the access narrow in scope and targeted.

fwiw, I’ve only done cli tools for agents to install in their own sandboxes, not access on remote machines.

This was just to avoid MCP infra, but the newer stateless protocol makes that much more trivial.

I do agree with you and was not aware of those reasons before, thanks for clarifying.

Completely agree, MCP is going to be a continuing topic for providing capabilities to LLMs for non-technical users. If you're in a tightly regulated enterprise environment asking your IT admin to enable CompanyX's MCP Connector in Claude is much more palatable than asking for a CLI tool to be installed as well.

It's about where the agent runs more than the users - hooking e.g. Datadog's Bits AI up with access to relevant repos in my company's GitHub org to find out what's driving the metrics its looking at likely wouldn't be a thing if not for GitHub's MCP server and OAuth, I doubt whoever admins that would bother if Datadog had to set up some completely custom thing.