News for hackers. Stuff that matters.
Re:Pure Bash Bible by jmnicolas on Thursday September 19, 2019 @08:21AM (#21015467)
I have a personal dislike for regexes and non human readable code, it gives maintenance headache. It's why I avoid shell scripts as much as possible.

The first example is not human readable, if the name of the function is a lie, I have no idea what this piece of code do :

  trim_string() {
    : "${1#"${1%%[![:space:]]*}"}"
    : "${_%"${_##*[![:space:]]}"}"
    printf '%s\n' "$_"
  }
Re:Pure Bash Bible by inimino on Thursday September 19, 2019 @09:08AM (#21015775)
I have a personal dislike for "non human readable" used as shorthand for "not immediately and easily readable by me".
Re:Pure Bash Bible by fhood on Thursday September 19, 2019 @11:32AM (#21017149)
I'm not sure you are being fair. I think any frequent user of regex understands how easy it is to produce expressions that are very difficult to parse.

I also think that a discussion about readability makes sense in a post about Bash. I don't personally program very often in Lua, Go or Ruby, but for the most part when I encounter code in these languages I don't find it very difficult to understand and modify. In contrast to this, every encounter with a significant amount of Bash seems to lead to a great deal of googling.

Re:Pure Bash Bible by inimino on Thursday September 19, 2019 @05:03PM (#21020745)
I haven't said anything about regexes here. What I wrote is that I find the shorthand "non human readable" specifically to be objectionable. Are regexes sometimes difficult? Sure. Are people who read bash also human? Yes, and that's my point. I hope that point is fair.
Re:Pure Bash Bible by deschutes on Thursday September 19, 2019 @11:29AM (#21017114)
The human part is a hyperbole. "This code is unreadable" implies "This code is unreadable for the intended audience."

Here's how I'd write that function: trim_string() { python3 -c 'import sys; sys.stdout.write(sys.argv[1].strip())' "$1" }

Re:Pure Bash Bible by ulysses on Thursday September 19, 2019 @12:20PM (#21017694)
"A collection of pure bash alternatives to external processes."
Re:Pure Bash Bible by munk-a on Thursday September 19, 2019 @02:00PM (#21018788)
It is a neat academic exercise to find these but I think that's sort of the point - why?

If you're building something that multiple folks will be reading and using just call out to external processes and make it easier to follow what's going on. It's neat to know how to do these things, but they're honestly a bit of a security hole because a large portion of folks using them will never comprehend the why and how and just assume it's doing the proper what.

Re:Pure Bash Bible by jascii on Thursday September 19, 2019 @02:46PM (#21019269)
Why? No external dependencies, your code can run anywhere you have a Bash shell. This also means a smaller attack surface.

Also readability: While some of the examples may be somewhat confusing to someone who doesn't have a lot of Bash experience, to someone who does it can be a lot more readable then feeding a string in yet another language to an external program and processing the output..

Re:Pure Bash Bible by munk-a on Thursday September 19, 2019 @03:30PM (#21019756)
Sed exists everywhere for a reasonably complete definition of everywhere - if it doesn't exist then you probably don't have access to Bash (and definitely not a fully featured one)
Re:Pure Bash Bible by jascii on Thursday September 19, 2019 @04:54PM (#21020658)
I have managed networks with 1000+ (microwave link) network devices that had Bash but no sed.. Sure was nice to be able to script maintenance on those..

Your definition of "reasonably complete" might not work for everyone else's use case.

Re:Pure Bash Bible by xelxebar on Thursday September 19, 2019 @08:43PM (#21022224)
Other people mention constrained environments, but tight loops are also a thing. If you know bash's parameter expansion well, it can make it really easy to write a one-liners that speedily process large amounts of output. Setting up and tearing down a whole process (e.g. sed) can cause an order of magnitude slow down, which is significant if you just want something quick and dirty.
Re:Pure Bash Bible by munk-a on Friday September 20, 2019 @12:05AM (#21023262)
I feel like... maybe? But quick or dirty should be enough - you could also build a python interpreter once to map over the data and then maybe write up some tests to confirm the functionality.

Every explanation of why to use bash here seems like it's got a whole lot of constraints on when it's a good to use - I've finished some tasks just in bash scripts but when I'm writing something for anything other than one time passes it seems like more of a maintenance liability than anything else.

Re:Pure Bash Bible by xelxebar on Friday September 20, 2019 @12:49AM (#21023413)
Meh. Choose your poison. As the trope goes, "No language is good at everything." Python is verbose compared to bash for many tasks, and the reverse is true as well.

IMHO, spending the time to actually learn some bash can really improve one's CLI life. Bashing on bash seems mostly like a tired ol' trope, "If it's in bash, it's bad."

The author of the original article really has written some pretty shell code!

Re:Pure Bash Bible by kazinator on Thursday September 19, 2019 @07:02PM (#21021610)
Useful in small embedded. Add twelve lines to a boot script, or bring in several new executables into /bin?

There are resource constraints even in large-ish embedded. Your main file system may live in a decent amount of flash space, but when the kernel is booting, it uses a tiny file system in RAM, which is pulled out of an initramfs image. There is a shell there with scripts.

The partition for storing the kernel image (with initramfs, device tree blobs and whatever else) might be pretty tight.

However, because a lot of these snippets depend on Bash extensions, they preclude the use of a smaller, lighter shell.

Re:Pure Bash Bible by Too on Saturday September 21, 2019 @01:17PM (#21036561)
Wouldn't a compiled binary be smaller and faster than an interpreted language? Assuming stdlib is already loaded.
Re:Pure Bash Bible by kazinator on Thursday September 19, 2019 @03:13PM (#21019577)
Problem: we're using trim_string in an initramfs image. Now we need Python3 in initramfs. Oops, kernel doesn't fit into the ARM board's flash partition any more ...
Re:Pure Bash Bible by clownpenis_fart on Thursday September 19, 2019 @03:17PM (#21019620)
lmao @ spinning up a python interpreter just to perform a trivial string operation
Re:Pure Bash Bible by inimino on Thursday September 19, 2019 @01:27PM (#21018458)
The problem is the direct logical implication of the hyperbolic statement "X is not human-readable" is that only non-humans can read X. Which is rude.
Re:Pure Bash Bible by munk-a on Thursday September 19, 2019 @01:58PM (#21018765)
I think this bible is an interesting attempt at avoiding call outs but... I agree with the parent - /\s(.?)\s/\1/ is just easier to read, even with character classes /[:space:](.?)[:space:]/\1/ than the trim example, it's actually a bit sad that string manipulation is the first section since string manipulation is such an inherently dirty task to do.

It is neat to discover functionality of bash I was unaware of by see folks push it to the limit though.

Re:Pure Bash Bible by serhart on Thursday September 19, 2019 @08:46AM (#21015624)
Just because you seem to be unfamiliar with bash doesn’t mean it’s unreadable. Almost any language will look cryptic if you don’t know it. That function is mostly just parameter expansion and very common in most bash scripts. I bet if you read the manual you would easily be able to figure it out. You just have to learn the language.
Re:Pure Bash Bible by mankyd on Thursday September 19, 2019 @09:14AM (#21015817)
I'm with jmnicolas on this. I've been using bash on and off for years now (approaching decades). I still get it very wrong almost always. It's the only language where this is a persistent problem for me.

Just yesterday I was struck by the difference between if [[ ]]; and if [ ];

I didn't even bother grokking the difference in the end. I simply found something that worked and moved on with my day.

Re:Pure Bash Bible by zests on Thursday September 19, 2019 @09:32AM (#21015968)
Single square bracket conditionals [] are the "older" (POSIX) version but do not have certain features that we would like to use in our shell scripts. Bash uses double square brackets to extend the functionality.

This is my understanding. Possibly not 100% correct but essentially correct enough for me to understand the reason/rationale for the difference.

Re:Pure Bash Bible by joombaga on Thursday September 19, 2019 @10:55AM (#21016761)
You got it! `test` is a shell builtin. `[` is basically syntactic sugar, and is a synonym for `test` with the additional requirement that the last argument must be a closing bracket `]`. `[[` is a shell keyword with more features.
Re:Pure Bash Bible by serhart on Thursday September 19, 2019 @09:34AM (#21015988)
I'm in no way defending bash as a language. There are lots of gotchas and weird constructs. I avoid bash too. It's just that trim function isn't that cryptic or "unreadable" if you know the syntax.
Re:Pure Bash Bible by pretendscholar on Thursday September 19, 2019 @09:39AM (#21016019)
I think the point is that it takes a lot longer pick the syntax up compared to some if/then/for/loop version of trim even if that would be way more verbose.
Re:Pure Bash Bible by serhart on Thursday September 19, 2019 @09:50AM (#21016124)
Yes, but I think that's antithetical to what languages like bash and perl are trying to do. Someone well versed in the language can do some pretty complex operations in a few keystrokes.
Re:Pure Bash Bible by pretendscholar on Thursday September 19, 2019 @12:21PM (#21017707)
which gets back to the original point that these languages have chosen to be more powerful over being more immediately readable. you have to know many more features of the language to understand what is going on compared to colloquially used constructs like if/then or for loops. If you understand english you are a lot closer to understanding the latter rather than the former, more domain specific syntax.
Re:Pure Bash Bible by serhart on Thursday September 19, 2019 @06:11PM (#21021253)
I don't agree at all. I don't find it less "readable". You just have to be used to it. If you understand English, German isn't that hard of a language to learn. Japanese would probably be a little bit harder to pick up though. Doesn't mean Japanese is unreadable
Re:Pure Bash Bible by cblades on Thursday September 19, 2019 @09:51AM (#21016129)
Maybe "unreadable" and "readable" aren't the best ways to approach this conversation.

It's certainly less readable than, say, my_str.strip()

Re:Pure Bash Bible by joombaga on Thursday September 19, 2019 @11:01AM (#21016823)
Are you comparing a call to a definition? `trim_string "$my_str"` is no less readable than `my_str.strip()`.
Re:Pure Bash Bible by mankyd on Thursday September 19, 2019 @12:23PM (#21017732)
I mean, I would say it is. Why are there quotes around the variable? What happens when I remove the quotes?

The quotes don't function like the parens. If this were a two argument function, you wouldn't put one pair of quotes around the whole thing. They're clearly transforming the variable somehow, but I'm not sure how and/or why they're necessary.

Re:Pure Bash Bible by serhart on Thursday September 19, 2019 @09:56AM (#21016187)
Sure, but bash only has certain language constructs. This implementation is using parameter expansion and some other built ins. It's definitely more complex than yours, but that's what bash has to offer.
Re:Pure Bash Bible by jigglesniggle on Thursday September 19, 2019 @09:48AM (#21016105)
I highly recommend running shellcheck on all of your bash code. It is what finally taught me the practical difference between [[ and [. There are some tests that will always pass in [, for example, but work properly in [[; one project never realized.
Re:Pure Bash Bible by mankyd on Thursday September 19, 2019 @10:00AM (#21016221)
My (personal) better recommendation is to avoid bash scripts whenever possible ;)

I generally tell people that, once a script is longer than ~100 lines and/or you start adding functions, you're probably better off with something like Python.

I know that's not a popular opinion with shell enthusiasts, but it's saved me so much frustration both in writing new scripts and coming back to them later for refactoring.

Re:Pure Bash Bible by cauthon on Thursday September 19, 2019 @10:33AM (#21016551)
> I generally tell people that, once a script is longer than ~100 lines and/or you start adding functions, you're probably better off with something like Python.

Agreed. My threshold is usually "when you want to start using arrays or dictionaries". I find bash best for file manipulation and running programs in other languages

Re:Pure Bash Bible by progman on Friday September 20, 2019 @09:58AM (#21026714)
Bash is a typical Unix tool, and Unix follows the KISS principle. Hence, also Bash scripts should be KISS.

If you need >100 lines of code your code is too complex in the script world, and you should split it into several scripts where each does one thing, and that one thing well. Usually, bash scripts are applied with pipes. A sequence like cat x | tr a b | sort >output is much more likely and easy to handle then a single script which does all these things.

KISS with Bash is a very different approach compared to other scripting languages like Python and Perl. There you can write long code easily and conveniently. However, things can get tough when larger scripts need to be maintained.

I consider the examples in the "Bash Bible" a collection of useful black boxes. It's fine if they just work. Regarding the "unreadable" trim_string for instance, if you have problems to understand that code, and you have to change something then you can simply write your own new trim_string script, even in Python or Perl if you like. Pipes work also well with them.

Update: Another advantage of bash and pipes over Python/Perl is that the Unix system can assign each script in a pipe to a separate thread. That means, simple bash scripts with pipes can work _much_ faster than single scripts in Python or Perl.

Re:Pure Bash Bible by tracker1 on Thursday September 19, 2019 @06:30PM (#21021398)
I tend to work in Windows, Linux and MacOS pretty regularly... so I tend to follow a similar approach (node instead of python though). Windows-isms under msys(git) bash in Windows just gets painful to coordinate at times.

I am curious if people will uptake PowerShell Core (though not much of a fan there either).

Re:Pure Bash Bible by shrimp_emoji on Thursday September 19, 2019 @12:07PM (#21017561)
I'm so glad I'm not alone in this.

I never even bothered to learn Bash properly because even that's difficult, and I figured it'd be "good mental money after bad". And, now that Python ships with all distros, I feel even less need to.

I do like Bash's range syntax though:

    for i in {0..5} ; do echo $i ; done
Like Ruby's. It's so nice! D: I lament Python's lack of it.
Re:Pure Bash Bible by yesenadam on Friday September 20, 2019 @08:58AM (#21026170)

    echo {1..5} 
is even nicer. Or

    echo {1..5}{1..5}{1..5}
Re:Pure Bash Bible by dec0dedab0de on Thursday September 19, 2019 @10:29AM (#21016511)
My rule of thumb is to use a real language if there is more than one if fi block.
Re:Pure Bash Bible by jascii on Thursday September 19, 2019 @02:49PM (#21019315)
So, what in your opinion constitutes a "real language", and why?
Re:Pure Bash Bible by dec0dedab0de on Thursday September 19, 2019 @03:29PM (#21019745)
a "real language" would be any programming language that is primarily designed and presented as a programming language.
Re:Pure Bash Bible by jascii on Thursday September 19, 2019 @05:39PM (#21021004)
Nice recursive definition..

The practice of programming in shell languages was well established when Bash was designed and Bash was definitely designed with that use in mind. So by your definition Bash is a "real" programming language.

Lisp on the other hand, was much more designed as a system and formal notation to reason about certain classes of logic problems.. So, Lisp is not a "real" language?

Re:Pure Bash Bible by dec0dedab0de on Thursday September 19, 2019 @09:40PM (#21022556)
Bash is primarily a shell, it even has the word shell in its full name.
Re:Pure Bash Bible by jascii on Thursday September 19, 2019 @11:07PM (#21022976)
And a shell can't be a real language? https://scsh.net/
Re:Pure Bash Bible by dec0dedab0de on Friday September 20, 2019 @09:22AM (#21026362)
a shell can't be a real language?

No, it can not.

Re:Pure Bash Bible by jigglesniggle on Friday September 20, 2019 @01:57PM (#21029472)
By "programming language" he probably means something that has robust flow control mechanisms and metaprogramming facilities.

You can definitely tell there's a different "feel" to bash and Tcl, Python, Perl, Go, etc, yes? Shell languages basically evolved out of batch processing languages that were meant to only run programs in sequence and it shows.

Re:Pure Bash Bible by jigglesniggle on Thursday September 19, 2019 @10:05AM (#21016264)
I agree, but I like phrasing it as "only run commands with substituted arguments." There's some more discussion on this below. :)

In the past I almost exclusively used Python, but I'm starting to like Go.

Re:Pure Bash Bible by cfors on Thursday September 19, 2019 @09:20AM (#21015862)
My disclaimer is that I love bash. I write way too many things in bash. But eventually I go back and refactor the scripts into Python (or lately, Go) because I am so sick of people refusing to touch my bash scripts.

String munging in Python is so much objectively easier to read than bash. I would say that it's because it is similar to a lot of the more popular languages than the sort of cryptic parameter substitution bash provides. And if that makes it easier to maintain some automation, then its worth the effort just to use Python IMO.

Re:Pure Bash Bible by fsaintjacques on Thursday September 19, 2019 @10:19AM (#21016396)
Python is a portability minefield. I'll spend more time trying to get the script/package run than reading the bash script.
Re:Pure Bash Bible by cfors on Thursday September 19, 2019 @10:27AM (#21016489)
Couldn't agree more, but honestly I think that is a completely overplayed issue. Just be explicit (This requires Python3.7) and have a requirements.txt file, maybe bundle with a `make install` command and move on.

If people can't figure that out... I don't know how you're going to expect them to read a cryptic bash script.

Don't get me wrong there is totally some good use cases for bash. Init-scripts come to mind when you don't want to lug around a Python VM in a lightweight container, for example.

Re:Pure Bash Bible by papito on Thursday September 19, 2019 @01:47PM (#21018652)
I don't think it's a matter of familiarity. Most of us here have used multiple languages, and I have no problem saying that the Bash syntax is atrocious.

"How do we close a statement.... errr. Let's spell it backwards".

Re:Pure Bash Bible by serhart on Thursday September 19, 2019 @07:31PM (#21021781)
How is it not a matter of familiarity? Just because it's not intuitive to you doesn't mean it's bad. I find anything not in an S-expression to be atrocious. Doesn't mean I can't learn and understand "horrible" syntax where I have to separate things with semicolons...
Re:Pure Bash Bible by papito on Friday September 20, 2019 @08:17AM (#21025794)
I am saying, we are allowed to have an opinion on it. You can be good at something and still acknowledge that is sucks.
Re:Pure Bash Bible by throwaway_se099 on Thursday September 19, 2019 @02:04PM (#21018824)
> "How do we close a statement.... errr. Let's spell it backwards".

That's on the original Bourne shell and its author's evident love for Algol 68. Wikipedia has a fine summary.

Re:Pure Bash Bible by fhood on Thursday September 19, 2019 @11:44AM (#21017282)
"Almost any language will look cryptic if you don’t know it."

Seems disingenuous to me. Java and Python are in a different class of readability than Bash or Perl.

Example: "abc" + "def" = "abcdef" vs "abc"."def" = "abcdef"

Re:Pure Bash Bible by lizmat on Thursday September 19, 2019 @11:59AM (#21017456)
> "abc" + "def" = "abcdef" vs "abc" . "def" = "abcdef"

Would be better comparison if you would use whitespace in the other case as well. Which then makes it just as readable.

Also. "0" + "42" would that be "042" or 42? It may be better readable, but the semantics are unclear.

Re:Pure Bash Bible by ofrzeta on Thursday September 19, 2019 @02:44PM (#21019244)
That's an argument for types. While it's more convenient to use a language without types it's more error prone. So I guess in the end strong typing wins.

  >>> 0 + 42
  42
  >>> "0" + "42"
  '042'
  >>> "0" + 42
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  TypeError: cannot concatenate 'str' and 'int' objects
  >>> int("0") + 42
  42
Re:Pure Bash Bible by peterwwillis on Thursday September 19, 2019 @10:12AM (#21016329)
Is the following a maintenance headache, or non-human readable?

  #!/bin/sh
  FOO=" some long string here "
  FOO="$( echo "$FOO" | sed -e ' s/^[[:space:]]//g; s/[[:space:]]$//g ' )"
This isn't "pure bash", but most of what I write in shell scripts isn't "pure bash". It's shell scripting: dirty, slow, easy, effective.

Like any 'language', it takes on the complexity you put into it. English is really complicated, but you can also use a subset of it with only 850, 1200, or 2400 words, and suddenly it's very simple and clear.

Re:Pure Bash Bible by jolmg on Thursday September 19, 2019 @01:24PM (#21018420)
> echo "$FOO" | sed -e ' s/^[[:space:]]//g; s/[[:space:]]$//g '

Your example fails when $FOO is "-n", for instance. Also the g modifiers are redundant in your example since there's only one beginning of line per line and one end of line per line.

I would instead write this:

  sed -r 's/^\s+|\s+$//g' <<< "$FOO"
EDIT: I think I see now what you probably thought would happen by using g, but no it wouldn't remove multiple spaces. So, you example also fails when $FOO is " x" (using 2 or more spaces at the ends).
Re:Pure Bash Bible by jwilk on Friday September 20, 2019 @03:13PM (#21030190)
Beware that -r is a GNU-specific option.
Re:Pure Bash Bible by jwilk on Friday September 20, 2019 @03:10PM (#21030165)
Unlike the original code, this strips whitespace from every line of input.
Re:Pure Bash Bible by seamyb88 on Thursday September 19, 2019 @10:10AM (#21016311)
> It's why I avoid shell scripts as much as possible.

This is all from somebody who has just written a linux user space. Perhaps the author avoids shell as much as possible. It just isn't possible. I doubt it, though. Bash is awesome.

Re:Pure Bash Bible by Too on Saturday September 21, 2019 @04:55AM (#21034253)
I only got to the first example and had a hard time deciding if the article was satire or not.

Let's use bash instead of a real programming language! It is super convenient! To do super fundamental things like trimming strings you just have to implement your own function with 40+ non-ASCII characters in a row! That example makes regexps seem like human readable and god knows if it is even correct or works portably across different versions of bash.

To be fair the article assumes you are first in bash and want to avoid launching subprocesses but for any production scripting that is the wrong hypothesis to begin with. A better approach would be to not start inside bash at all. Just do your scripting in a higher level language where basic ABC stuff like string, list, number and error handling are already there for you. Even if that trim_string function and everything else from the article would be provided in some bash-bible-std-lib it wouldn't even come close to what's available in say Python for example, and you still have to wrestle the syntax and other obscurities like -a (or -e) meaning file exists.

While there are some good examples in there if you are stuck in bash, this article was more of a 100 reasons not to use bash to me.

Re:Pure Bash Bible by 5Qn8mNbc2FNCiVV on Friday September 20, 2019 @04:04AM (#21024267)
And it doesn't have to be. There should be trust that the function does what it says. If I use `printf` and don't understand how it's working internally it doesn't matter to me because I know what it does and trust it.

These scripts/snippets for me are just an unofficial extended standard lib

Re:Pure Bash Bible by 0x0aff374668 on Friday September 20, 2019 @01:29PM (#21029162)
"maintenance headache."

Then stop maintaining code from languages you don't understand. This is frustrating. I've seen solutions on the comments that call PYTHON! Are you effing kidding me? PYTHON ??

Granted, BASH docs aren't particularly succinct, but shell scripts are an absolute necessity in the OS world.

It's pretty simple: If you don't understand shellcode, don't maintain an OS, or rather, don't expect to be accommodated for lack of knowledge for something that's been standard for 30+ years.

Re:Pure Bash Bible by dredmorbius on Thursday September 19, 2019 @05:29PM (#21020933)
My inner pedant compels me to reply that these are not in fact regexes, but Bash parameter expansions.

https://www.gnu.org/software/bash/manual/html_node/Shell-Par...

My inner pedant has no comment regarding readability of Bash parameter expansions.

Re:Pure Bash Bible by kamaal on Friday September 20, 2019 @12:29AM (#21023335)
On the contrary regexes were invented because you shouldn't have to write 500 lines of code for that which can be done in two lines.

If we didn't have regexes we would have 100s of if/else's scattered all over program logic. That would be more hard to handle than the regex itself.

Re:Pure Bash Bible by chapium on Thursday September 19, 2019 @10:02AM (#21016241)
Perhaps it shouldnt be defined as what the regex can do, but which unit tests its able to pass.
Re:Pure Bash Bible by Koshkin on Thursday September 19, 2019 @11:22AM (#21017047)
It might be worth it to learn it though, even if just for fun.
Re:Pure Bash Bible by soheil on Thursday September 19, 2019 @11:42AM (#21017250)
Do you have a personal dislike for math too?
Re:Pure Bash Bible by imihai1988 on Thursday September 19, 2019 @01:57PM (#21018754)
those are not regexes though, are they ? apart from the [:space:] part