That's not the idea of shell. The idea is that I write programs in Python, JavaScript, R, and C++ regularly, and about 10 different DSLs (SQL, HTML, etc.) And I work on systems written by others, consisting of even more languages.
I need a language to glue them together. A language to express build automation and describe deployed systems. Most big systems consist of more than one language.
Shell is the best language for that, but it's also old and crufty, with poor implementations.
When you program in shell, gcc, git, pip, npm, markdown, rsync, diff, perf, strace, etc. are part of your "standard library".
>> The point of shell is that it interoperates freely with all programs and all programs share a language.
Also doesn't mean people should stop trying to come up with these new shells. May be some "new" shell will catch up and tomorrow's "all" programs will be compatible with that shell. For example, now on Mac OS X, now ZSH is the default shell.
There are more young people who know Perl or Tcl or Lisp than know Crush. I've never understood this reasoning - "Hardly anyone knows X. Let's replace it with something nobody knows." At most, you're making an argument about a lack of sunk cost.
> May be some "new" shell will catch up and tomorrow's "all" programs will be compatible with that shell
Think of it like transitioning to life in a wheelchair. You can't "catch up" with your house - it just wasn't designed for wheelchair use. Renovating all the narrow corridors, random steps, tight angles and vertical reaches probably costs more than rebuilding.
In this comparison, the mobility mode is your shell.
Because there's a value proposition. Just like TCL, Perl, and Lisp had value propositions over other available languages. It's fine if you don't see the value proposition in Crush but progress sometimes forces us to learn new things instead of sticking with what we have.
If I’m being honest, you’re not adding anything constructive at all here - it seems you’re in a competition with yourself as to how much you can degrade Crush for whatever reason.
You act like it’s a choice between Crush and Perl. People don’t have to learn Crush. People don’t have to learn Perl. Will picking between the two have any substantial effect on one’s career outcomes these days? Probably not, as candidates will have already learned the important languages for getting a job anyway.
But it is cute to couch it in a complaint about not being constructive; points for shamelessness.
By that logic no one should make anything new to replace the incumbent.
In case of Perl, PHP came and it replaced Perl as mainstream web development language.
In sysadmin stuff where Perl used to be mainstream, now significant chunk is taken away by Python.
These days JavaScript appeals to younger people more. Times change and with it the tools people use also change.
There is no guarantee that any new project will succeed but you won't know unless you try.
You're really bringing up a separate point which is "should we be reinventing the wheel?", I think that's a fair point, but it's not a response to the original commentator.
I wouldn't hold my breath for something else to replace sh/bash as the default shell on the major nixes within this century. Pessimist? More like realist :-)
The "ecosystem" of programming languages is healthy. There are many options, some to master, some to consider. But not knowing how to use a tool -- especially a widely-used and powerful tool -- is no badge of honour or discernment.
but what if people don't know how to learn?
instead of getting wrapped up in all of this 'knowledge preservation' nonsense, shouldn't we just mimic what they are already doing and assume it to be better, because the activity occurs in a year number that is higher than the previous behavior pattern?
now that would be real progress.
or, not.
Societal results sadly noted.
I don't know perl and I'm not young, but I do understand and appreciate the role perl -e plays as an extension of bash. When I run into an issue where I need more than bash, but less than python, I'll search around SO for a perl -e command for what I need.
This is often how people decide to learn a tool: they have a problem for which it is the best solution. They get some exposure to the capability and slowly expand on their knowledge.
What do you mean by can't? If it misses some mode of interaction, that can be always added.
Sometimes the inconsistent solution will be simpler by moving the complexity to the choice of the mode of interaction. Sometimes not.
"ls" lists files. Sure. Simple interactive command. But, thinking about this, "echo " also lists files. "ls" provides information about files. Sure. Simple interactive command. But, "stat" specifically provides information.
So, I would reach for
stat --format "%n %a"
To get name and octal permissions. You are correct, and wrong at the same time. Easy enough to accomplish -- but using * instead of ls. Note that the "space in filename" becomes the big issue (use %N in stat format).
So your conclusion is 100% correct. Thank you for your post and insight
ls -l | awk '{print $3}'
ls -l | cut -f 3 -d ' 'I think the fact that you can already use this with most programs and it is never less powerful than bash seems like good indication that the system doesn't need to be reworked around this shell. It fits in cleanly to existing paradigms.
Perl code is often written once and then runs for years without issue. I have on occasion fixed broken Perl scripts despite having practically no experience with the language. It wasn't a problem at all, because your average maintenance job is "something was moved" or "we're switching out X for Y", but not "we need to rewrite this complex system that is a Perl script for some reason".
I would intuitively agree with you as there are hundreds of unnoticeable perl scripts doing their job, but as someone with moderate experience with the language, I can say that when I did maintenance tasks, they were mostly the second kind.
Tcl is horribly, terribly under-appreciated. It is one of the great might-have-beens of modern computing, almost as much as is Lisp.
And I think Lua is more elegant than TCL overall. TCL's stringiness really gets clumsy in certain types of programs, and it's a much larger target than Lua overall.
Not to say that TCL is bad, but I'm glad that dict-based languages prevailed over TCL's string-based-ness
# ls | grep foo
This involves two programs written in nicer language (C).
# ls | rg foo
This involves two programs written in nicer language (C & Rust).
Feel free to come up with JS program that does something well and/or is usable in pipeline. I think you're missing the point of the shell.
I especially see this being useful in my CI environment which has a handful of functions, some json and yaml parsing (using jq and python respectively), etc. It's a little cumbersome at the minute but not so much to switch from bash and the rest of the project is JS anyway so it would be good to be consistent.
> useful in my CI environment
For sure. I've been meaning to play around with getting a docker image with the shell set as the default. It's definitely a use-case I had in mind.
A lot of people seem to have negative opinions about Tcl (maybe because it's more of a LISP than its syntax seems to imply), but as someone who is just discovering it, I'm shocked at how powerful it is for how unpopular it seems to have become. It's like some kind of unexpected cross between Bash, LISP, and (with Tk) Visual Basic. There's something about it that's just kind of fun.
I think some of the pushback might be because people tried to use it to build large applications and failed. Languages like Java and (dare I say it) Visual Basic were more comprehensive and had good IDE support.
Yet Tcl tends to be forced on undergraduate programmers, who probably just learn to loathe it. I lost several toes to it myself decades ago, and haven’t really felt inclined to pick it up again.
And before you know it you have a buggy monster written in a batshit insane language that nobody is going to want to touch.
If your shell is at least vaguely sane, the situation is a lot better. I don't really get why you would argue against something that fixes the many many many mistakes of Bash.
I see many insane constructs that accidentally work. I also know how much time it would take to explain expansion rules. And people don't even want to hear it because it shakes their simplistic (and inconsistent) understanding.
So yes, I hope we'll see something like crush replacing Bash. But I don't see how it will happen.
U should fix ur computer rather then spreading FUD.
Also its clearly stated that copy of pwsh is intention on both projects.
Nushell is still one man show just like crush so they need decade or so to get to pwsh level.
Its nice to make stuff and offer alternatives but lets keep perspective here.
My computer runs just fine. Do you really think pwsh is fast?
Vim is slower for example but rarelly anybody compalain about it given its power.
Windows wasn’t designed around the command prompt, it wasn’t designed around shell utilities. When I call “cat”, it’s not important that I’m writing those three letters, they are a library way of calling a set of file open/read OS functions. When I write “ping”, it’s not the ping tool I care about, it’s the network api calls it does behind the scenes. The Unix/Linux shell is then seen like a “Python standard library” of ways the user can control the OS, and it’s a hugely inconsistent, undesigned, ad-hoc agglomeration of names and parameters and bodges, which cheats out on dealing with the complexity and instead offloads that straight on to the user in a user hostile way.
What powershell tries to do for Windows is make a somewhat designed, consistent, and more user-approachable set of wrappers around the Windows OS system calls, along with guidelines and support for vendors to build in that style, a human interface that is introspectable, built to be programmable, but still composable and modular.
The objection that trying to do such a user-focused thing on Unix makes a “useless” system is weird because that - a user interface to tools/library of OS call patterns - is what the shell is. In what way is “resolve-dnsname” pretending to be one with the system that dig and nslookup aren’t? They’re names which trigger a pattern of network traffic and show a result. In what way is piping text from dig to sed “real” and piping resolve-dnsname to format-list “pretend”?
In what way is running “docker compose” from sh real, but running it from zsh is pretend?
In what way is it a better design to have (a pile of shell tools and Python library wrappers and Python wrapping shell tool calls), instead of (a Python based shell with transparent shell tool calls)?
Like, “I want what I’m familiar with, too many things to relearn”, or “too many things are awkward in this approach (cough PowerShell)” are reasonable objections, but “trying for a more consistent, more human friendly, interface with more high level language constructs at one’s fingertips makes a thing useless to me” feels heel dragging or Stockholm syndromeish. Presumably you don’t object to there being a choice of multiple command line tools to do a task? Why object on principle to a choice of other OS “front end experiences”? Is there any version of things which you can imagine as being both “better” (less arbitrary and inconsistent than shell) and “not pretend”?
> “The point of shell is that it interoperates freely with all programs and all programs share a language.”
What language? Some treat piped bytes as bytes (e.g. piping a binary file), others as ASCII, others as UTF-8. All support different parameter options, names and values and ways of specifying them. Many handle output formatting in different ways, many trigger internal states from environment variables. If the point is a common shared language, the point doesn’t seem to have been achieved from a user perspective. And if the point is that “I have to suffer a system that’s tied to a shared language with an AIX box in a bank basement last touched by a human in 1997” the point is a bad one. I use my computer a lot more than I use AIX and I have more computing power to spare on human comforts and conveniences.
The UNIX philosophy was to write a solid system by taking most of the hard problems in kernel space and kicking them into user space. This is not a slur, this is a description. Especially at the time, it wasn't at terrible solution. I kind of think of it the same way as many programming languages have observed that "the standard library is where things go to die"; the code in a system that is hardest to change, well, it doesn't change! So any bad decision that ever gets in there is almost permanent. Part of the reason UNIX won is that it enshrined fewer bad decisions in the kernel than its competition did. I think you could even argue it had more bad decisions overall than its competition, at least towards the beginning; it's just that far fewer of them were enshrined in the kernel.
This is also the philosophy of the pipe. It disclaims all layout and moves that responsibility on to the consumers. There is no "shell language", at all. This is, simultaneously, a big strength and a big weakness.
One aspect of that is precisely that the strength of this approach tends to inhibit anyone who tries to beat it with something like Powershell. All Powershell had to be was better than any shell on Windows. This was not hard. For something like Crush, to take off it has to be better than any existing shell on UNIX, and as annoying as unstructured pipes may be, the ecosystem as a whole still has a lot of positives about it, because it's been tuned for 50+ years. It's going to take a lot of work to match that, and so far, it doesn't seem like any of these projects are achieving take-off velocity.
Arguably, the best thing to do would be for all these projects to get together and try to define a standard UNIX-level non-textual rich serialization of some sort, using lessons from Powershell and arguably things like D-Bus to do it, because none of these projects individually seem to be on a success trajectory.
Who cares about "taking off"? If it's useful for 10s of users, it has taken off.
Consider xonsh (https://xon.sh/). I don't think it will ever be "mainstream" the way bash or zsh are. I doubt it will ever come preinstalled in any major Linux distribution. But it's used by enough and has enough momentum that I don't worry it will die. That alone is enough for me to use it as my primary shell. I don't care that xonsh will never "take off".
It doesn't have to be the goal of the project or the author, but there are real advantages to achieving a takeoff, even to the users of a product.
What is the one thing that GNU ls does well, by the way?
Backwards compatibility with itself, probably.
If you came away from my post with the impression that this system produces a lot of elegant software, well, I must have screwed up in my writing somehow. What it produces is bodges upon bodges, which are then polished by years of ramming into the corner cases and putting in another bodge to fix them. There is actually something to be said for this design methodology, and it's not all bad, but it rarely produces elegance.
Now, at least GNU ls doesn't send or receive email or something. A quick scan of its man page doesn't show many any features grossly out of line with its primary task. But yeah, it has a lot of features for sure, many of which nominally overlap with tasks that could be done by other tools (sort, ignoring backups, -D ("generate output designed for Emacs' dired mode", lol), quoting styles...).
I feel much the same way about Python. Very little of its philosophy expressed back around 2000 or so is in effect today. And there's no one release you can point at and say, "this is it, this is when it stopped being simple", yet, clearly, today it is no longer simple.
The "UNIX philosophy" gets cargo cult by younger generations thanks to a couple of statements in a few worshipped books, yet there is hardly any UNIX clone post V6 where it was actually followed upon.
* Preserve compatibility with popular shells like bash.
* Address some of the broken parts in a way that makes the world better. Quote handling is the original sin of *nix shells. If somebody fixed that you would have my full attention.
Otherwise I can just stick with Bash + Python3.
In order to preserve compatibility, you must handle quotes, namespaces, etc in the same way as Bourne-derived shells, because they will be expected to run scripts that use nested interpolation, or string declarations without quotes, or whatever. Shells that use a more robust syntax, like Fish or Powershell, can't ever be compatible.
http://www.oilshell.org/release/latest/doc/simple-word-eval....
You can try it right now. Turn on 'shopt -s simple_word_eval' to get the better behavior.
Or turn on the group 'shopt -s oil:basic' and get even more better behavior.
When those options are off, Oil runs some of the biggest shell scripts in the world. It's the shell that's most compatible with bash by a mile.
https://github.com/oilshell/oil/wiki/Shell-Programs-That-Run...
Scripting languages like Python, Ruby, Perl, etc., have dealt with upgrade many times. It has not always been pretty, but the alternative is stagnation.
https://news.ycombinator.com/item?id=24086213
And yes I drafted a blog post called "Unix Shell Should Evolve Like Perl 5" which says exactly what you are saying:
http://www.oilshell.org/blog/2020/07/blog-roadmap.html#the-u...
And Oil has already done it. It needs to be faster, but that's an inevitability based on progress so far.
Why Use Oil? http://www.oilshell.org/why.html
If Apple shipped some Swift-powered shell and all of their management documentation started using it, I'd bet a non-trivial portion of administrators would start to use AppleSwiftShell (although I do deeply appreciate that's a severely flawed comparison since macOS already has a sane functioning shell, but it's the best I could think of as a huge company that owns the platform pushing such a concept down)
Its doing good because it is good and people started to notice once MS propriatery chains went down.
String interpolation isn't done via quoting but via the format method on the string type, e.g. `"foo {} baz":format "bar"` generates the string "foo bar baz".
Similarly, to execute a command on a remote host, pass in a closure as an argument to the `remote:exec` command, like `remote:exec {1+1} "example.com"`, which will return the value 2.
I find that when quoting becomes a problem it usually means that the wrong solution has already been picked. If you're doing double quoting, you have already lost.
The power and usefulness of a given language is often the ability to serve multiple goals at once. Python as tool is cool specifically because of a large-ish but accessible-ish library covering a everything from regex-ex to a web-server.
This project might not serve the goal of writing high level programs and interacting effortless with the command line/system. But I don't one should automatically dismiss the goal of making stuff-glued-together look like a coherent whole.
I know that Crush currently doesn't interact very well with external commands, but I don't see where that is an inherent restriction in the design rather than a missing feature.
This is why I started my shell. Eventually it will have the ability to run both POSIX and other shell languages in the same program.
Assuming everything is a table is overly restrictive. Assuming everything is text is the UNIX way. It's then the job of the terminal to render it.
This presupposes that the Unix shell is the way it is because of careful, considered, forward-thinking design, rather than being a haphazard heap of mouldering hacks upon lazy corner-cutting upon quick-n-dirty bodges that should’ve been thrown out and replaced decades ago.
I agree that trying to improve the *nix shell is a bad idea… because the _Unix shell itself_ is a bad idea. Fifty years is long enough. Instead of bending over backwards for that obsolescent garbage heap (for which no-one will thank you, as we see), study it, learn its lessons, and draw a line under it.
Then go build a clean, modern shell architecture from scratch, using modern tooling and techniques, unhobbled by the myriad defects and deficiencies of the old. It’ll take a tenth of the time and be ten times better (if not, you’re still doing it wrong).
See also:
https://www.google.com/search?q=sunk+cost+fallacy
https://homes.cs.washington.edu/~weise/uhh-download.html
--
p.s. As for driving early adoption, don’t even think of trying to win over the Unix old guard (who will only bring regressive practices and complaints with them), but sell it hard to the new generations of users who are accustomed to modern UI/UX standards and much less impressed by what the “Old Unix” dogpile has to offer.
Then your environment is useless to me. Because I have better programming languages which, despite being able to interact with the system and its commands, don't pretend to be one with the system. As a result, the core high-level language constructs mesh better, because they're not trying to serve two goals at once. It's not possible for something that's not a 'real' shell (including fish, zsh, even csh) to integrate with the system the way a real shell does, because the system itself was designed around the shell.