I’m trying it right now and the initial results seem promising. The openai whisper voice to text works nearly flawlessly and everything else seems to be falling into place. I’m thinking of using local gpt or similar models to classify and then distribute requests to more programmed scripts.

Are you just padding input windows to 30 seconds, or have you come up with a smarter way to handle "streaming"?

Just padding, there's a built-in function which just does it. That is definitely in the realm of "problems for much later", the bits that interest me are what to do with the input, and 30 seconds to talk is more than enough to accomplish what I'm thinking about.

A physical button for input is what I'm thinking as a side project, I like that much more than always-on listening. There are also lots of much less heavy weight voice to text which could do streaming listening waiting for a key phrase which when found could start recording and send the output up to the better model. I don't know what they are, but I know they exist.