It's usually ~two seconds or so for the first line to come out of OpenAI. You can see the example video in the github, though I think that was little slow.

If you have big atuin file though, creating indexes for session and cwd are good ideas so we can get the request out quickly..

edit: but much faster than searching or asking LLM for command line parameters instead

Copilot is highly usable and I'm pretty sure it works the same way.

Copilot also runs a small model on the users machine to quickly provide simple predictions.

Interesting, I didn’t know that.