This is a great point. Not that far. We also snapshot the desktop for "slow" non-streaming updates to the UI. We could push these into Claude itself to act on or describe or whatever.

That's the next move :-D

For web apps, I'd guess that many of us already do that via Playwright or other MCPs. I'd bet there are people doing something similar with desktop apps too.

unfortunately there is nothing like that for the desktop apps.

Even mobile app devs have https://github.com/mobile-next

Why not https://github.com/hrrrsn/mcp-vnc ?

The streaming architecture is designed for exactly this - we originally built it for autonomous agents that need persistent development environments. The missing pieces are mostly integration work (mapping Claude's tool use format to our desktop APIs). Would be very interested to hear if others are working on similar integrations - the combination of LLM coding agents + real desktop environments feels like it unlocks a lot of interesting workflows.

I did that a year ago, I imagine it would work better today.