I can't stop but see here parallels between cmdline arguments vs. environment variables for programs, and keyword arguments vs. dynamic binding for functions in a program (particularly in a Lisp one).
That is, $ program --foo=bar vs. $ FOO=bar program
seems analogous to:
(function :foo "bar")
;; vs.
(let ((*foo* "bar"))
(function))
I suppose the latter argument applies to env vars as well - you can accidentally pass values differing from expectation because you didn't know a parent process changed them. The former doesn't, because processes don't have a "signature" specifying its options, at least not in a machine-readable form. Which is somewhat surprising - pretty much all software written over the past decades follows the pattern of accepting argc, argv[] (and env[]), and yet the format of arguments is entirely hidden inside the actual program. I wonder why there isn't a way to specify accepted arguments as e.g. metadata of the executable file?
Speaking of using both direct and dynamic arguments, a common pattern in (Common) Lisp is using dynamically-bound variables (so-called "special" variables) to provide default values. Since default values in (Common) Lisp can be given as any expression, which gets evaluated at function-call time, I can write this:
(defun function (&key (some-keyword-arg *some-special*))
...)
when I build command line utilities and I think about the way that they'll be used, I tend to use configuration files for things that will change very slowly over time, and environment variables as a way to override default behaviors, and command line arguments to specify things that will often vary from invocation to invocation. In fact, most of the time, I use the environment variables either for development/testing features that I don't really intend to expose to most users, or for credentials that don't get persisted.
it's never occurred to me to use environment variables as a primary way to configure an application. I'll have to noodle on that for a while. My gut says that it's enough of a deviation from the Unix convention that I probably won't use that.
It's not completely unheard of. Checking the man pages for a few I remember being affected:
* `ls` looks at LS_COLORS
* `grep` looks at GREP_OPTIONS, GREP_COLOR, GREP_COLORS, as well as the not-grep-specific LC_ALL, LC_COLLATE, LANG, etc, and POSIXLY_CORRECT
* `find` also looks at LC_%, LANG, POSIXLY_CORRECT as well, plus TZ and PATH for some of its other options
Looks like the program-specific ones LS_% and GREP_% can be overridden by options directly on the command, but the others don't have such options.
(Makefile wildcards because asterisk keeps doing italics)
Me too. My point about env vars was limited to key/value options. I'm happy to use positional arguments when names are irrelevant, e.g. `ls foo/` is better than `DIR=foo/ ls` or `ls --dir foo/` or `ls --dir=foo/` or whatever, since the latter just adds noise and increases the chance we'll get something wrong.
I'm also happy to use arguments for flags which are either present or absent, where values are irrelevant, e.g. `ls --all` rather than e.g. `ALL=1 ls` or `ls --all=yes` or whatever. In this case the presence of a value is even more hamful, since the program might only be checking for the variable's presence, in which case `ALL=0 ls` would be misleading; plus we'd have to guess whether the value should be 0/1, yes/no, y/n, on/off, enable/disable, etc.
This is also a good example of why having GNU/Linux as one's first OS instead of a real UNIX®️ is so toxic.
There's also the issue that typo'ing "foo" as "fooo" will silently fail. Okay, that's a simple example, but some tools have "PROGRAMNAME_LONGDESCRIPTION". Being in all-caps also makes it hard to spot typos for me. You generally want your env vars to be long, so ensure they're unique.
I don't think I disagree with your post, but this particular point seems to be trivially solved by prefixing the variable name with the program name?
IOW: `env | grep '^GIT_'` to see what you're passing to `git`.
I use 'find' so much that my muscle memory would hurt if it changed, but it's actually a really interesting example. Its use of arguments to build a domain-specific language is a cool hack, but pretty horrendous; e.g.
find . -type f -a -not \( -name \*.htm -o -name \*.html \)
key=foo val=bar jq -n '{(env.key): env.val}'
jq -n --arg key foo --arg val bar '{(env.key): env.val}' jq -n --arg key foo --arg val bar '{($key): $val}'When writing automation routines env vars are often better form my experience
Tools that will be manually run by a user or admin then cli options are better
For instance, the fact that the aws(1) cli tool and the AWS SDKs by default take IAM creds from AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, etc. means that you can write programs like aws-vault (https://github.com/99designs/aws-vault) which can acquire a temporary AWS session and then execute another command with the creds injected in the standard AWS_... env vars. For instance:
aws-vault exec prod-read -- aws s3 ls
aws-vault exec stage-admin -- ./my-script-that-uses-the-aws-sdk-internally
Some thoughts/remarks:
- If we don't change a variable/value then it's immutable. Some languages let us enforce this (e.g. 'const'), which is nice, but we shouldn't worry too much if our language doesn't.
- We're violating 'single source of truth' if we have some parts of our code reading the env var, and some reading from a normal language variable. This also applies to arguments through.
- Reading from an env var is an I/O effect, which we should minimise.
- We can solve the last 2 problems by reading each env var once, up-front, then passing the result around as a normal value (i.e. functional core, imperative shell)
- Env vars are easy to shadow, e.g. if our script uses variables FOO and BAR, we can use a different value for BAR within a sub-command, e.g. in bash:
BAR=abc someCommand
let foo = 123
bar = "hello"
baz = otherCommand
quux = let bar = "abc"
in someCommand
E.g. I had a really weird bug in a script, git seemed to break for no reason, until i figured out that some other script had exported GIT_DIR which pointed to the git top level directory. The name of the variable isn't bad if want to save that directory. But git uses GIT_DIR as the location of the .git directory it should look at when it is defined.
Using environment variables is kind of like using global state, it can often cause weird behavior at a distance without a clear path of influence if you don't know exactly which environment variables are used by all your scripts.
I write a lot of tools in Haskell, where 'getEnv' is an "IO action" whose type prevents us using it inside most code. The simplest way to handle this ends up with a "functional core, imperative shell" model, e.g. most of my CLIs have the form (where 'interact' takes a pure String -> String function and uses it to transform stdio):
main = do
foo <- either error id . parseFoo <$> getEnv "FOO"
interact (doTheThing foo)
doTheThing :: Foo -> String -> String
doTheThing = ...
It's not universal though; you can't in C or shell scripts for example (although, if you really want to, it's not hard to assign a global). I never had the need to inspect commandline arguments beyond the main() function though.
Very simple and removes all the extensive command line clutter, especially if many settings stay the same.
I think environment variables are not very user friendly and not that visible.
If commands insist on taking options via a config file, my preferred way to call them is to generate the config in-place via process substitution, e.g.
someCommand -c <(printf 'foo\n\tbar="%s"\n\tbaz="%s"\n' "$BAR" "$BAZ")
Using arguments for key/value pairs allows invalid states, where the key is given but not the value. It can also cause subsequent, semantically distinct, options to get swallowed up in place of the missing value. They also force us to depend on the order of arguments (i.e. since values must follow immediately after their keys), despite it being otherwise irrelevant (at least, if we've split on '--'). I also see no point in concatenating options: it introduces redundant choices, which imposes a slight mental burden that we're better off without.
The only advice I wholeheartedly encourage from this article is (a) use libraries rather than hand-rolling (although, given my choices above, argv is usually sufficient!) and (b) allow a '--' argument for disambiguating flag arguments from arbitrary string arguments (which might otherwise parse as flags).