> The shell cannot automatically deduce appropriate completions when possible.

I don't think this even _can_ be possible for a large number of programs.

For some small number of programs it _might_ be possible, if somehow (ignoring how for now) you were exposed some standard interfaces like getopt or argparse and didn't have to care about argument ordering.

But a ton of other programs, with extremely complicated interfaces, simply wouldn't be possible. And these include commonly used programs.

Auto-completing awk would require a full language parser at the least, for example.

Another, ffmpeg, has a dizzying array of options, and the order of those options can completely change the intent of the command, and what options are allowed to follow without being ignored silently.

Even if we had perfect interface detection, we could only probably generate a subset of auto-completion options because that particular problem might not _always_ be solveable.

You raise a good point about solvable completion, and one that I agree with. However, automatic completion support doesn't have to cover every use case. It only needs to be good enough. Complex applications can fall back to providing their own completion scripts.

One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked.

And as for completion for the command's arguments, the command is only ran once to generate the list of completion candidates.

Note that there are projects that already do exactly this like [1] .

[1]:https://github.com/kislyuk/argcomplete

> One point I'm not clear on is why you think such a scheme wouldn't be feasible for a large number of commands. Completion wouldn't be invoked until the fist argument (the command's name) is complete just like how bash doesn't scan the system's completion directory until it knows what command is being invoked.

Simply because some command pipelines are extremely complicated. For example, with ffmpeg, some flags can disable earlier flags, or re-enable them. Completion isn't a simple left-to-right thing. There's other programs that can be similar.

> Completion isn't a simple left-to-right thing.

Sorry, I still don't understand why that's relavent. It's probably my fault for not being clear enough. Please be patient with me as I try to further elaborate:

If the command is in charge of both generating completions and also for parsing arguments, wouldn't it be aware of all inner-dependencies between flags? Wouldn't such awareness make it possible to generate candidates based on previous flags?

I'm not familiar with ffmpeg, but let's assume that there was another similar program called ggmpeg. Let's assume that it accepts the flag '--input-format' and then either the flag '--repeat-video-range' or '--repeat-image-frames', depending on whether the format is an video or an image.

That is, only the following flag sequences are valid:

    ggmpeg --input-format=video/webm --repeat-video-range=.. ...

or

    ggmpeg --input-format=image/png --repeat-image-frames=.. ...
    

Let's also assume that ggmpeg signals to the shell that it is responsible for generating its own completions by passing the flag --_completion to it.

Now, if the user types the following command

    ggmpeg --input-format=image/png --repea[CURSOR]

and then presses tab, thereby invoking the shell's auto completion routine, which in turn ultimately executes

    "ggmpeg" "--_completion" "--input-format=image/png" "--repea"

wouldn't ggmpeg have all the necessary information to only suggest '--repeat-image-frames' and not '--repeat-video-range'?

Ok, here's a simpler example that might demonstrate why it can't always be solved.

    ffmpeg -i input.avi -metadata author=shakna output.mp4

What metadata keys are allowed, if at all, actually depends on the output file. If I'd chosen to make an ogv instead, then that metadata flag would become disabled, because it doesn't accept most sets of metadata.

I can't complete:

    ffmpeg -i input.avi -metadata a[TAB]

Because the entire set of CLI commands isn't simple left-to-right solveable.

However, if I do:

    ffmpeg -i input.avi -metadata author=shakna output.mp4 output.ogv

That means I'm outputting one file with metadata, output.mp4, and one file without it, output.ogv. So the flag is conditionally enabled or disabled.

I can now see what you meant by completion not being left-to-right solvable. Thanks for taking the time to lay out a clear example.

While completion isn't be solvable in general, surly we can agree that its usefulness (when solvable) merits making it more readily available to users.

Having the shell implementation automatically deduce suggestions from the command itself (when available/possible) would go a long way towards increasing availability in my opinion.

Some shells do parse manages and use other tricks to auto-determine completes. Though they're not POSIX compliant shells so you'd have a to retrain a little there.

I'm interested to know more about that ELF / --_complete trick though. Do you have any more details? (when DDging the only result I can find on the topic is your comment)

> (when DDging the only result I can find on the topic is your comment)

You weren't kidding at all.

After spending more time than I'd care to admit searching for the library, I couldn't find traces of it anywhere on google, ddg, or bing. I relented and digged whatever information I could find in my history.

Revisiting these projects, it's painfully obvious that I have jumbled a few projects together. Sincere apologies for the unintended goose chase.

This project [1] is the one that introduced the `--_completion` flag. It registers itself as a default completion function, but blindly and dangerously tries to run any command it gets with this flag. (This project is completely de-indexed from search engines and I'm not sure why).

Like [1], project [2] registers itself as a default completion function, but checks the first 1024 bytes for the magic string "PYTHON_ARGCOMPLETE_OK". It seems to be relying on argparse for completion.

I couldn't manage to find the project that was scanning elf sections. I'm not entirely sure if it was relying on elf sections specifically or if the fact that it was scanning elf sections was only an implementation detail.

[1]: https://github.com/dbarnett/python-selfcompletion [2]: https://github.com/kislyuk/argcomplete

Awesome. I really appreciate you taking the time to research that for me. Some great material for me to read through. Thank you

I'm only guessing this is what PowerShell was supposed to enable. I've never used it but I gather programs are supposed to offer enough metadata to do this?