I know to do this intuitively, but I have no idea why.

It tells the shell utility that any remaining arguments are not options, but instead files or whatever the script might process. You know, in case someone makes a file called -rf.

But not all shell utilities follow this particular convention

Yes. It’s more of a convention. If a shell utility isn’t following it then it won’t mean what we think it would mean.

Also - it helps a lot when a utility accepts a path that may or may not contain hyphens.

Most of the commonly used ones do, so it's easiest to just always do it and then remember the two or three utils that don't like it.

Yes. The famous echo on Linux systems does not have it and therefore it's impossible to print the string "-n o p e", because -n will be interpreted as an option.

It does if single or double quotes are used, right? Which would be necessary (or preferred to multiple backslashes) quite often.

No, the quotes are not seen by the program. The program receives a list of strings, it does not get the information about whether and how those strings were originally quoted in the shell. Programs can also be directly called with lists of strings as in execve, so often it does not even make sense to ask if the arguments were quoted or not.

Quotes live on a different level of abstraction.

> No, the quotes are not seen by the program. The program receives a list of strings, it does not get the information about whether and how those strings were originally quoted in the shell.

With quotes the program will receive a single argument -n␣o␣p␣e instead of multiple ones -n, o, p, e. At least it works on the machine here:

    ]$ echo "-n o p e"
    -n o p e
    
    ]$ /bin/echo "-n o p e"
    -n o p e

Yes, I think there was some misremembering here. The nontrivial thing is to print out -n itself with echo. For example, echo doesn't treat "--" specially, so "echo -- -n" prints "-- -n".

Note that this is true for POSIX sytems but not e.g. for Windows. There the program receives the command-line as-is and is responsible for parsing it into an array. There are two different standard functions to do this parsing for you (with slightly different quoting behavior) but you could also create your own that requires options to not be quoted.

echo is not portable anyway, use "printf %s STRING" or "printf '%s\n' STRING".

Yes, that's what I use. Sometimes I still get tempted to use echo because there's less typing...

It terminates argument parsing, so anything following it that starts with a hyphen will not be treated as an argument.

    $ echo 'hack the planet' > --help
    $ cat --help
    cat: illegal option -- -
    usage: cat [-belnstuv] [file ...]
    $ cat -- --help
    hack the planet
    $ rm -vf --help
    rm: illegal option -- -
    usage: rm [-f | -i] [-dIPRrvWx] file ...
           unlink [--] file
    $ rm -vf -- --help
    --help
    $ cat -- --help
    cat: --help: No such file or directory

My eyes have been opened. Thank you!

It's worth it just to watch the frustration of a junior when they try tacking more arguments on the end of a command.

A great opportunity to teach the importance of reading the whole command before trying to modify it.