It’s more about sophisticated replacement for tab completion.

I can just run Llama locally for that, no VC required (and indeed there are plugins which do exactly this already). Llama and other local LLMs are being optimized to run on the CPU on less than 1 GB of RAM so it's not as if only those with beefy GPUs can do so.

Agreed and the part I’m missing is these plugins which do this. Any suggestions or recommendations would be interesting to hear. Which terminals and which plugins etc.

Thankfully it doesn't run it, it just inserts the commands into your prompt. It's kinda nice but I've also been using the terminal for like 12 years, so I don't find myself relying on it very often.