This reminds me of easy motion for vim or ace-jump for emacs.
Do you think it would be possible to have an on-demand contextual hat decoration?
Like you say “show hats words” and only words get decorated with hats and you pick one. It would allow you to maybe show hats only on square brackets or only on function arguments, etc. I find the number of hats with colors a little bit hard to distinguish; should they were contextual, they would require fewer or no color.
Do you map voice commands to keyboard shortcuts available in vs code, or directly via the apis? (Not sure if there is a difference in the end).
Now I wish for a cursor less plugin on the IntelliJ platform.
The reason that the hats are always present is that the way to code faster by voice than be keyboard is to speak fluently, minimising pauses, the way we speak regular human languages. If we had to say a command and then wait for the hats to appear, that would break the chain.
Re mapping, we use something called the "Command server", which allows us to use file-based RPC to run commands in VSCode. That way it is easy to send more complex commands, which are required by Cursorless
IntelliJ support is definitely one of the most requested features; once I'm done rewriting some of the core engine I'll probably take a swing at that. Here's the issue that tracks extracting cursorless into a node.js server so that it can be used by other editors: https://github.com/pokey/cursorless-vscode/issues/435
I think it's a lot faster than keyboard / mouse, mostly because of how little moving of the cursor you have to do.
Could be I was slow to begin with, not super efficient with vim or emacs.
Also, "editing" is the fastest part for me, due to "bring" and "change". So little movement.
For the former ("programming"), there are many commands that can be used to rapidly output code. For example, a user can say "funky hello world" to get something like
function helloWorld() {
}
But if you're trying to edit code that already exists, it can be a challenge to do so without a mouse and keyboard, in part because you need to do a lot of navigation.
I personally believe Cursorless solves that problem better than a keyboard and mouse, but I have to imagine I'm a bit biased on that point :)
I also have a friend who is a gifted programmer who lost his ability to type about a decade ago; he has put together an open-source software stack to help: http://www.cs.columbia.edu/~dwk/
Of course this doesn't really answer your question. But it's a hard problem, and you're basically forced to become a power user to reliably interact with your PC.