And somehow people think mastering this careless design is something to be proud of.
Can anyone name any other popular piece of software that is that relaxed about irreversibly destroying your data?
I jest but I do agree that Bash et al have their warts. Seriously obscene warts that only make sense when you look at the language from the content of 40 year old systems. But these days there are other options if you want the power of a command line but without (most of) the warts of traditional shells.
I didn't know about SawStop before, so your comment sent me on a wiki-hunt.
"Gross negligence" seems like a huge understatement for all the rationalizing and lobbying manufacturers did to avoid having to use safety technology.
https://en.wikipedia.org/wiki/SawStop#Opposition_from_trade_...
At least in zsh, you can prevent such mistakes by
setopt NO_CLOBBER
Also, from bitter experience I've forced myself to get in to the habit of never typing:
rm *
cd ..
rm foo/*
In Bash it’s
set -o noclobber
set -Chttps://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
Only because people refuse to read the specification. The POSIX Shell & Utilities volume is exceedingly clear and straight-forward. (I vehemently disagree w/ its characterization in that paper.) It helps if you use the frames-based HTML version to make it easier to navigate and explore the various utilities, and especially to see how the syntax specification is broken down--while that section is short, it's difficult to appreciate without explicitly seeing how the subsections are related.
I only code in POSIX shell because I really can't be bothered to remember all the minutiae of other shells, almost all of which provide strong POSIX compatibility, Bash especially. I've never lamented the lack of arrays; but I regularly wish that POSIX would adopt ksh-style coprocesses so dash would have them. The shell is really all about stitching together processes (including subshell contexts), and coprocesses fill a conspicuous gap in functionality.
Bash has a different implementation of coprocesses that doesn't seem to be used much, and is very limited. I believe there can only be one coprocess active at a time, which is very un-shell-like! I got an e-mail once from someone who was using them, and I felt it could be achieved by other means.
I have a good idea coprocesses in Oil which basically is a startup time optimization. The coprocess will basically take argv over a unix domain socket, and have other commands to change the process state. It solves "the problem of VMs and containers that start slowly".
https://www.oilshell.org/blog/2021/04/release-0.8.9.html#wha...
Unix domain sockets are already used for the headless shell in Oil (more coming on the blog about that)
https://www.oilshell.org/release/latest/doc/headless.html#im...
Basically, you start a coprocess using the "|&" operator:
somecommand |&
cat <&p
"p" always references the most recently created coprocess, so to juggle multiple coprocesses you have to dup "p" to explicit descriptors:
exec 4>&p
somecommand2 5<&p |&
coprocesses provide a simple construct for cases where you would otherwise have to manually create a named pipe using mkfifo. The need for this pattern comes up surprisingly often, IME, yet because it's so messy--now you need to worry about /tmp races, cleaning things up, etc--people rarely bother, myself included. From the perspective of a shell it's a relatively simple construct to implement. The semantic benefit/cost ratio is exceptional.
This basic functionality is complemented with extension options to the read and print (non-standard) builtins: -p reads from or writes to the latest coprocess, and -u the specified descriptor number. I haven't used coprocesses much because I can't rely on them, but I don't think these are strictly needed. Maybe they were required for proper buffering in early implementations (or even current ones), where redirection to and from builtins didn't utilize the same buffered I/O context? That seems more like an implementation detail.
ksh93, bash, and some other shells provide the process substitution construct, "<()", which can provide similar capabilities to coprocesses in many cases. But I'm pretty sure anything you could do with process substitution you could do with coprocesses, making the latter the more powerful construct. coprocesses seems like the critical missing primitive from the traditional Unix shell model, and would round it out. (Of course, the sky is the limit for evolving the Unix shell model in new directions, as Oil has shown.)
ksh88 had coprocesses so I'm curious why it wasn't included in POSIX Maybe because bash lacked an implementation? Or incompatible quirks between ksh88 and ksh93 related to stdout inheritance across coprocesses?
You are right it has some warts, but if a file is really that absolutely critical I make a .bak copy of it before I start my work. The shell has its limitations but so do all tools and the power it gives me is unmatched.
As for other popular software there is plenty out there but other popular software out there destroys something far more valuable to me. My time.
This seems to miss the point completely, unless you back up your entire hard disk before running every command. The whole problem is it's easy to nuke things you weren't intending to touch, accidentally.
The user-hostility of the shell is not a required feature to achieve its power. All of the advantages you cite could be had without being a terrible footgun with human factors mired in 1970s computing culture.
I am not a carpenter and context switching is expensive when it comes to thought work (probably because it’s still just typing, lacking the mechanics of swapping a hammer for a screwdriver).
Anyway, the point is by now, 20+ years into an IT career, I should have at least ONE option that can handle these needs without flip flopping through an entire operating systems “tools”.
The desktop metaphor is not the only real world model computing needs to get rid of. I want an interface that can do the job, not a million “tools”.
echo "2 + 2" | bc -lIt's a job control language after all.
The pipes and redirects, and the instant access to the file system is also very powerful.
Sure it could/should be improved massively, but I'd say it's still the best tool for a significant amount of pretty common use cases, it doesn't just suck at everything.
wget http://server{1..5}.com/file.txt >> /tmp/data.txt & wait && wc -l /tmp/data.txt
Hence everything has been stuck with the archaic limitations and oversights of posix shell or gnu bash as linux distros took over and largely standardized on it.
There have been numerous alternative shells and some significantly improve on the situation, unfortunately they've been largely ignored. Default shell choice has proven very sticky.
I do agree that it's practically impossible to write robust and safe scripts in bash/posix shell. You have to jump through many hoops if arbitrary user input is being handled. Frankly nobody should be writing generalized tools using such shells today, unless they have a very good reason to self-impose the constraint.
99.99% of everything is internet connected and auto-updates. New distro versions come out every 2 years.
There's no real reason why bash couldn't have been evolved into a sane language with sane parsing and with modern constructs. By sane, I mean... read this:
https://www.oilshell.org/blog/
At least these articles:
https://www.oilshell.org/blog/2021/01/why-a-new-shell.html (especially this one)
https://www.oilshell.org/blog/2016/10/20.html
https://www.oilshell.org/blog/2016/10/28.html
https://www.oilshell.org/blog/2016/11/06.html
If freaking Javascript could "use strict;" and turn a page, bash could have definitely have done the same.
It's just that there's no real commercial interest and FSF doesn't have the manpower (or desire) to do this.
https://www.oilshell.org/blog/2020/07/blog-roadmap.html#the-...
It turns out that bash's "shopt" can be co-opted for this, and that's exactly what Oil does. For example, after 'shopt --set parse_paren', you can do this:
if (x > 0) { # parens are now Python-like expressions, not subshell!
echo "$x is positive"
}
Oil is written in a way to make breakage less likely: with principled parsing and a detailed code representation using algebraic data types. I'm surprised at how much language evolution this style has enabled. It makes me optimistic about the project's future!
Although we still tons of work to do with the C++ translation -- writing a shell is hard either way!
True, but that's not a good reason for such a used application. It has been around for 40 years, I think it literally has trillions of runtime hours. It's used on billions of systems, most likely.
It's exactly the place where you'd create a super comprehensive test suite that covers <<everything>> (see SQLite) and then re-architect it to make it robust.
I wonder if it's due to lack or money or due to something else.
> https://www.oilshell.org/blog/
I think the point is that, it did evolve, as your link shows—and there are lots of other sane alternative shells, too. The problem isn't that they don't exist; it's getting people to adopt them in sufficient numbers.
Just wanted to point out that Hacker News runs on FreeBSD[0] :) Some people like the BSDs, and would appreciate it if everybody stopped designing for the Linux monoculture (well, monoculture if you disregard Windows and macOS).
[0] https://docs.freebsd.org/en/books/handbook/introduction/
"What HN uses" doesn't say much about tech trends :-)
Edit: You probably want to use Netflix as a better example. I think it's used by their CDN layer.
The funny thing is that there about a dozen ways to spell it, like $(dirname $(readlink -f $0)). Since the shell itself doesn't provide this functionality, people come up with a lot of workarounds.
In Oil it will simply by $_this_dir, e.g. "source $_this_dir/mylib.sh" for relative imports. (Whenever a variable is silently mutated by the interpreter, akin to $?, Oil prefixes it with _).
https://github.com/oilshell/oil/issues/587
I solicit feedback on every Oil release: http://www.oilshell.org/blog/tags.html?tag=oil-release#oil-r...
so let me know what you think of the fixes: https://github.com/oilshell/oil/wiki/Where-To-Send-Feedback
Dealing with filenames is fixed in Oil: http://www.oilshell.org/blog/2021/04/simple-word-eval.html and with QSN:
http://www.oilshell.org/blog/2020/10/osh-features.html#safe-...
Again, if you don't think this actually solves the problem, leave some feedback. Oil is the most bash-compatible shell by a mile, so you might need to use it someday :)
(The theory is that in MOST situations you don't get to choose your language, just like you didn't choose C, C++, or shell. It's all the inertia of compatibility.)
Oil has named parameters: http://www.oilshell.org/release/latest/doc/idioms.html#use-n...
I know about the long file list problem, but I don't think I've ever run into it, mainly because I use xargs, which batches up commands correctly. If there is some other situation where it comes up, I'm interested.
Insane combination of grep / awk / cut: Oil has eggex which compiles to ERE to help you write egrep and awk patterns: https://www.oilshell.org/release/0.8.11/doc/eggex.html
You get syntax errors when you write the pattern (at parse time), not when you run it.
Crucially, we don't "rewrite" grep and awk. We just make them easier to use (optionally). There is a smooth upgrade path and you can retain your knowledge while forgetting about some sharp edges.
> Personally I find it amazing that people invented this thing, then spent decades tripping on the same quoting and filename handling issues and not finding that to be a good reason to fix it.
Yes I quoted David Korn complaining about the quoting problem in the early 90's, which was closer to Unix's invention than we are to the early 90's.
https://www.oilshell.org/blog/2019/01/18.html#slogans-to-exp...
The reason we still have it is the inertia of compatibility, and the fact that nobody really owns shell. Even though people rightly complain about Google's web stewardship, HTML5 was a great improvement and cleanup. They paid people to fix HTML, and it worked to a large extent.
Some food for thought: A Generation Lost in the Bazaar https://queue.acm.org/detail.cfm?id=2349257
https://github.com/oilshell/oil/issues/941
(This probably won't happen for a long time though, unless someone really wants to contribute and own it!)
Totally agreed. A shell script exceeding that's about 10 lines of code becomes so complex to read, interpret and improve that even though it may perform faster I would rather get the job done in a language that I can understand better when I come back to this 3 months later.
Although ... on that note I don't think I'd ever choose Perl ha ha.
This question does not really make sense. There need not be any script, or there may be many at the same time. What exactly do you want? Imagine, for example, that you are piping the output of some commands into the shell.
Sure, but that is irrelevant.
When there IS a script, you should be able to get its full path much easier, that is the complaint.
Doesn't need to act differently in what it does.
But it does need to report it, because e.g. you might want to bundle data/assets/config relative to the script, and want to be able to run it from wharever and have it still be able to find it.
In the later case, does it mean that it is not possible to write reusable code based on that feature? In the former case, what should be done when sourcing is done recursively? Can this be used to defeat file location, for instance, by generating a script in /tmp and sourcing another file from it?
Secondly, it's super frequent that scripts I write or that others write need the folder they're executed in as an argument.
99% of scripts I've seen aren't meant to be piped to. They're just launched as simple, dumb, standalone "programs".
SCRIPT_DIR="${0%/*}"
the same applies with "combination of grep, awk and cut". awk contains almost all the functionality of grep, sed, and cut, so any pipeline of awk plus one of the others is almost always unnecessarily complicated. it would be like complaining "in python, string handling is so hard. look, you have to do import re; for x in re.sub(...).match(5).split(' ')[3:-5]: lst.append(x)". of course if you make it overly complicated then it will be overly complicated.
there are also too many footguns. spaces and newlines are trivial to handle in 95% of scripts, as long as you quote every expansion. as long as you never ever write cmd $file and always write cmd "$file", that solves virtually all whitespace problems. quotation marks in filenames are never interpreted in a special manner in Bourne-like shell unless you use eval, which has the same pitfalls as any other interpreted language (see python above).
SCRIPT_DIR="$(dirname $0)"
Just because we wish something weren't so, doesn't make it as we wish it.
No program can do this portably.
But it's possible for an interpreter to tell the script it's interpreting where it lives, i.e. argv[1]. For example, Python does this in a pretty straightforward way.
How do you find where the script itself is? Because if the script comes with files, we want to find those when the script gets called from somewhere outside the current dir. Stack overflow suggests this as a first approach:
Then follows with 7 lines of arcane looking code containing a loop for a better solution. That's a ridiculous hoop to jump through for a common need.Dealing with filenames? Hope you enjoy pain. Spaces, newlines and quotes are a horrible pain to deal with properly, and regularly cause trouble.
Dealing with correctly passing parameters? More of the same.
Dealing with long file lists? You run into the command line length limit.
Obtaining some incredibly common system parameter? Some insane combination of grep, awk and cut, probably.
Personally I find it amazing that people invented this thing, then spent decades tripping on the same quoting and filename handling issues and not finding that to be a good reason to fix it.
These days if it's more than 5 lines of code, or if arbitrary filenames are involved I go straight to Perl or Python, because the amount of completely preventable pain is drastically reduced.