A device like the Rabbit r1 looks promising. You don't technically need to run inference on the same device. We're still a few years away from making this efficient and small enough to run locally, but there is a lot of potential for the next generation of AI assistants to revolutionize how computing is done, and have mass market appeal.

I'm surprised that Apple hasn't jumped on this yet. All the building blocks are in place to make a revolutionary device possible. It just needs to be very polished, and have great UX, which is what Apple typically excels at.