The exception being if it's a Python app in the first place, then it's fine since it can be assumed you need to be familiar with Python to hack on it anyway. For example I write a lot of Ruby scripts to include with Sinatra and Rails apps.
For a general purpose approach however, everybody should learn basic shell. It's not that hard and it is universal.
It's not a matter of familiarity with linux shell. It's a matter of wasting time debugging/implementing stuff in shell that is trivial to implement in any modern scripting language.
But even JSON and (X|HT)ML you'd be amazed how often a simple grep can snag what you want, and regular expressions are also mostly universal so nearly everybody can read them.
Python wasn't our main language (the app was in C++ and QML which is basically java script). But we both knew Python and it's the easiest language for these sort of things that we both knew and that comes with the system.
My main conclussion is - I'm starting with python the next time.
Life's too short to check for the number of arguments in each function.
Any complex or cross-platform C++ projects will need a scripting language in addition to shell and a build system.
Ha! The Oil shell does that for you :)
https://www.oilshell.org/release/0.8.0/doc/idioms.html#use-p...
(It's our upgrade path from bash)
[0]: I'm sure there's a jq style library for python somewhere but it's definitely not the norm.
and yes, parsing XML or JSON in "linux shell" is very easy. For JSON use jq, for XML use xml2 or xmlstarlet.
Fortunately for me a 10 whole lines of sed, awk and some trimming and a for loop turned my json into a CSV to pass back to a legacy system. No curl or json modules necessary. That being said, if I needed anything more complex Python would be my default.
I really like Ruby as a replacement shell language because its syntax for shelling out is very succinct and elegant. Python is a bit more verbose but a lot more explicit. Either one is a step function better than shell. You get a real programming language.
Let's extend your analogy to a logical conclusion. Should you write something in a shell because it's "not that hard and universal" or should you insist on using a programming language that lends itself to writing maintainably? If not, we should have no problems with PHP and COBOL, no? But we do.
Use the right tool for the right job. If you're not sure what that is, don't hesitate to pull out a glue language. Python, Ruby, JS -- whatever you need to get the job done. Your shell should be just that -- a shell, not the core.
And after all that it broke once it moved to a AIX machine since it was not POSIX compliant.
Maybe it's obvious to you, but when I cannot even figure out if an if statement will work, something is horribly wrong.
I don't even use python and I can just about guarantee it's easier for someone who knows neither.
Fabric is the best for this: you put a `fabfile.py` (name inspired by Makefile) in the project root and add @task commands in there. Specifically the `fab-classic` fork of the Fabric project https://github.com/ploxiln/fab-classic#usage-introduction (the mainline Fabric changed it's API significantly, see `invoke` package).
Fabric can run local or remote commands (via ssh). Each task is self-documenting and the best part it that it's super easy to read... almost like literate programming (as opposed to more magical tools like ansible).
if __name__ == '__main__': args = docopt.docopt(__doc__)
means nowt to me.
In fact I don't really understand your post and I've written a lot of python.
No language has a simpler api to executing a cli command than bash itself. By definition.
__name___ == '__main__'
simply tests whether this Python file is being executed from command line.It’s one of the ubiquitous Python idioms that doesn’t actually follow Python’s zen of “readability”.
Perhaps I shouldn't have tried to make two points in one post; I wouldn't advocate using docopt or a main function for a simple script, I was more making the case for how easy it is to add proper parameter parsing when you need it; that is easier to remember than what you'd end up writing in bash, which is something like:
PARAMS=""
while (( "$#" )); do
case "$1" in
-a|--my-boolean-flag)
MY_FLAG=0
shift
;;
-b|--my-flag-with-argument)
if [ -n "$2" ] && [ ${2:0:1} != "-" ]; then
MY_FLAG_ARG=$2
shift 2
else
echo "Error: Argument for $1 is missing" >&2
exit 1
fi
;;
-*|--*=) # unsupported flags
echo "Error: Unsupported flag $1" >&2
exit 1
;;
*) # preserve positional arguments
PARAMS="$PARAMS $1"
shift
;;
esac
done
# set positional arguments in their proper place
eval set -- "$PARAMS"
I think you've got to write a lot of bash before you can remember how to write `while (( "$#" )); do` off the top of your head; the double-brackets and [ vs ( are particularly error-prone pieces of syntax. while getopts "ab:" OPTC; do
case "$OPTC" in
a)
MY_FLAG=0
;;
b)
MY_FLAG_ARG="$OPTARG"
;;
*)
# shell already printed diagnostic to stderr
exit 1
;;
esac
done
shift $((OPTIND - 1))
Yes, I realize it's more difficult to support long options (though not that difficult), but the best tool for the job will rarely check all the boxes. Anyhow, the arguments for long options are weakest in the case of simple shell scripts. (Above code doesn't handle mixed arguments either but GNU-style permuted arguments are evil. But unlike the Bash example the above code does support bundling, which is something I and many Unix users would expect to always be supported.)Also, I realize there's a ton of Bash code that looks exactly like you wrote. But that's on Google--in promoting Bash to the exclusion of learning good shell programming style they've created an army of Bash zombies who try to write Bash code like they would Python or JavaScript code, with predictable results.
I agree that a cli framework is often easier to use than bash, but it also is a dependency. I think everyone should use what he’s familiar with.
Things that are true "by definition" are not usually useful.
"No language has a simpler api [...] [b]y definition" only if by "executing a cli command" we mean literally interpreting bash (or passing the string through to a bash process). But that's never the terminal[0] goal.
The useful question is whether, for the task you might want to achieve, for which you might usually reach for the shell, is there in fact a simpler way.
It may very well be that the answer is "no", but support for that is not "by definition".
[0]: Edited to add: ugh. Believe it or not, this was not intended.
Usage: db_backup.py INSTANCE
then just go with it.Is it mostly the Make-style task decorators for managing the project task list that you consider a win vs. native subprocess.run? Are there other features that you find valuable too?
(Back when the original fabric was written subprocess was much less friendly to work with, of course).
# Option 1 (no pipes)
import os
for line in open("/etc/passwd").read().splitlines():
fields = line.split(":")
if fields[0] == os.getlogin():
print(fields[-1])
# Option 2 (Cheating, but not really. For most problems worth solving, there exists a library to do the work with little code.)
import os
print(os.environ['SHELL'])
# Option 3 (pipes, not sure if the utf-8 thing can be done nicer somehow)
import subprocess
username = subprocess.check_output("whoami", encoding="utf-8").rstrip()
p1 = subprocess.Popen(["grep", "^" + username, "/etc/passwd"], stdout=subprocess.PIPE)
p2 = subprocess.Popen(["awk", "-F", ":", "{print $NF}"], stdin=p1.stdout, stdout=subprocess.PIPE)
print(p2.communicate()[0].decode("utf-8")) subprocess.check_output("grep ^$(whoami) /etc/passwd | awk -F : '{print $NF}'", shell=True, encoding="utf-8").strip()
I didn't test that particular line, but in general this is how I execute shell pipelines in Python.In bash, you have to use commands for most things. In Python, you don't.
For example, `curl ... | jq ...` shell pipeline would be converted into requests/json API in Python.
Edit: The equivalent of above would be:
#!/usr/bin/perl
my $shell=(getpwuid($>))[8];
print "$shell\n";
Though it does have simple syntax for pipes as well. pwd.getpwnam(getpass.getuser()).pw_shellMy last gig, I stumbled onto shebang, which allows you to invoke shell scripts in any scripting language from the command line.
https://en.wikipedia.org/wiki/Shebang_(Unix)
Major face-palm-slap. I'm certain I've read about shebang many times. But never realized that it's general purpose.
One nice thing about it is the overloading of pipe operator to allow easy creation of pipelines and/or redirection (plus it is cross platform, so you don't depend on the system having a certain shell).
Okay, but what about software engineers?
Shell is simple and straight forward, and can bring you quite far for most stuff. Only if you start using complex datastructures it makes more sense to use a mature language like python.
listing = `ls -oh`
puts listing
puts $?.to_i # Prints status of last system command.
Some equivalent Python 3.7+ would be
import subprocess
listing = subprocess.run(("ls", "-oh"), capture_output=True, encoding="utf-8")
print(listing.stdout)
print(listing.returncode)
As an experienced Ruby programmer I imagine the code you provided looks like a no-brainer. To a Ruby neophyte the backticks, $? sigil, and .to_i method don't strike me as intuitive (but maybe they would be do someone else). We may just have to disagree about the nicety of syntax.As cle mentioned[1], the strongest criteria is
> what would the majority of people on the team prefer to use?
and I'd agree that ultimately that's what makes the most sense.
just put this on the first line #!/usr/local/bin/node instead of
#!/usr/local/bin/python
Since 3.5 added `subprocess.run` (https://docs.python.org/3/library/subprocess.html#subprocess...) it's really easy to write CLI-style scripts in Python.
In my experience most engineers don't have deep fluency with Unix tools, so as soon as you start doing things like `if` branches in shell, it gets hard for many to follow.
The equivalent Python for a script is seldom harder to understand, and as soon as you start doing any nontrivial logic it is (in my experience) always easier to understand.
For example:
Combine this with `docopt` and you can very quickly and easily write helptext/arg parsing wrappers for your scripts; e.g. Which to my eyes is much easier to grok than the equivalent bash for providing help text and requiring args.There's an argument to be made that "shell is more universal", but my claim here is that this is actually false, and simple Python is going to be more widely-understood and less error-prone these days.