News for hackers. Stuff that matters.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by akira2501 on Sunday March 12, 2023 @03:01PM (#35124620)
So, '>' isn't for redirection anymore. How does one do that?

    open foo.txt | append "world"| save --raw foo.txt
Oh.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by pxc on Sunday March 12, 2023 @03:32PM (#35125055)
* * *
Re:Nushell.sh ls | where size > 10mb | sort-by modified by andrepd on Sunday March 12, 2023 @04:03PM (#35125482)
It is much much better to have things be actual commands (which have a uniform syntax, and are `--help`able) rather than obscure sigils like $(#!@$).

Fish is amazing!

Re:Nushell.sh ls | where size > 10mb | sort-by modified by pxc on Sunday March 12, 2023 @04:15PM (#35125637)
Many of these next-gen shells like Nushell are clearly (and sometimes explicitly) inspired equally by Fish and by PowerShell, and it shows. It's a brilliant combination!
Re:Nushell.sh ls | where size > 10mb | sort-by modified by amelius on Sunday March 12, 2023 @05:12PM (#35126370)
I think the problem with this approach is that it breaks down at some point, and then your only option is to rewrite the entire thing in a real scripting language.

I.e., there is no graceful scaling path.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by stingraycharles on Sunday March 12, 2023 @05:34PM (#35126579)
Isn’t this the problem with older shells as well? Eg you have to use e.g. awk or sed to deal with more complicated text processing. My understanding that the biggest difference between powershell (and nushell) and the older bash-like shells is that they deal with structured, object-like data rather than text. How would things be any different when “things break down” vs an older shell?
Re:Nushell.sh ls | where size > 10mb | sort-by modified by jrumbut on Sunday March 12, 2023 @09:56PM (#35129739)
One of the nice things about bash is that it interacts very nicely with Perl and Ruby.

I wonder how these other shells work with the backticks and such in Perl/Ruby. Theoretically there is room for very cool interoperability but I wonder how it goes in practice and if any nice tools and patterns are being developed.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by phist_mcgee on Sunday March 12, 2023 @11:57PM (#35130832)
Since this is nushell and perl and ruby are now starting to be considered 'old, is that so much of a problem?
Re:Nushell.sh ls | where size > 10mb | sort-by modified by jrumbut on Tuesday March 14, 2023 @04:47AM (#35148806)
It's a path to scaling. You can basically dump a shell script (I assume including nushell) into your .pl/.rb file with a little punctuation and then slowly refactor it to a language that can handle greater complexity.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by pxc on Sunday March 12, 2023 @05:27PM (#35126514)
I suppose I'm more optimistic. From my point of view, structures pipelines and some of the goodies that come with it (like better error handling), can make a shell language into a perfectly capable scripting language. I think the big weakness for these new shells is in the small sizes of their library ecosystems. But PowerShell shows what a language of this kind can do with a sizeable ecosystem and some OS integration.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by ilyt on Sunday March 12, 2023 @06:46PM (#35127406)
Should be doing that anyway, bash is terrible for actual programs.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by coldtea on Sunday March 12, 2023 @07:05PM (#35127755)
As opposed to what? Regular shells which break down even quicker?

And there's no hard constraint that would require this to "break down at some point". A well designed shell could also be a "real scripting language", all-in-one.

But the horrible CP/M and POSIX legacy of shell design doesn't help with that.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by inciampati on Monday March 13, 2023 @12:48AM (#35131222)
Fish is amazing. And. It was missing critical features like subprocesses for so long (10 years?) that I had to keep passing it by. No subprocesses!
Re:Nushell.sh ls | where size > 10mb | sort-by modified by pxc on Monday March 13, 2023 @10:32AM (#35136202)
* * *
Re:Nushell.sh ls | where size > 10mb | sort-by modified by jntrnr on Sunday March 12, 2023 @03:55PM (#35125362)
Or:

  open foo.txt out> bar.txt
Re:Nushell.sh ls | where size > 10mb | sort-by modified by codetrotter on Sunday March 12, 2023 @03:55PM (#35125363)
Historically we would discourage shell users from

    cat somefile.txt | whatever
and instead tell them to use either

    whatever < somefile.txt
or, if the command accepts input file names as arguments then use those like

    whatever somefile.txt
or

    whatever -i somefile.txt
etc

But perhaps in fish, “useless use of cat” is not so useless and would be recommended?

(I mean aside from the fact that they seem to recommend their “open” command, and so they probably prefer their own “open” over “cat”.)

Re:Nushell.sh ls | where size > 10mb | sort-by modified by mtlmtlmtlmtl on Sunday March 12, 2023 @04:06PM (#35125511)
For some reason I never liked to do it the "right way". First of all it doesn't seem to actually matter.

But most of all the cat way just aligns with my mental model more. Data flows left to right, if you catch my drift.

It also makes it easier to add arguments to the end if re-running it.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by Arnavion on Sunday March 12, 2023 @04:31PM (#35125856)
>But most of all the cat way just aligns with my mental model more. Data flows left to right, if you catch my drift.

Using `<` doesn't change that model. You can write `< somefile.txt whatever`. I always write my commandlines as `<in_file cmd1 | cmd2 | cmd3 >out_file`

(Though annoyingly this doesn't work for feeding input to while loops. `< <(echo a; echo b; echo c;) while read -r foo; do echo "$foo"; done` is invalid syntax; it needs to be `while read -r foo; do echo "$foo"; done < <(echo a; echo b; echo c;)`)

Re:Nushell.sh ls | where size > 10mb | sort-by modified by ilyt on Sunday March 12, 2023 @06:49PM (#35127472)
> Using `<` doesn't change that model. You can write `< somefile.txt whatever`.

If you showed that without context it would look like some prefix/lispy notation so no, you are just showing how bad it is

Re:Nushell.sh ls | where size > 10mb | sort-by modified by Arnavion on Sunday March 12, 2023 @07:18PM (#35127966)
"This thing I don't understand reminds me of another thing I don't understand, so it's bad I decided."
Re:Nushell.sh ls | where size > 10mb | sort-by modified by ilyt on Sunday March 12, 2023 @08:26PM (#35128804)
The poster above complains it does not read left to right well.

So you give more examples that do not read left to right well.

Please take your smol brains takes elsewhere

Re:Nushell.sh ls | where size > 10mb | sort-by modified by mtlmtlmtlmtl on Sunday March 12, 2023 @09:10PM (#35129240)
I thought it was an informative response. I certainly learned some stuff about shell I didn't know before. I'm still gonna use cat because it's simpler to me.

I think your snark is unfounded here.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by cerved on Monday March 13, 2023 @01:29AM (#35131481)
doesn't it work if you put it after the while but before the read?
Re:Nushell.sh ls | where size > 10mb | sort-by modified by Arnavion on Monday March 13, 2023 @12:05PM (#35137623)
No. That will execute `< <(...) read` anew for every loop iteration, which in this case will cause it to print an endless stream of `a`s.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by tomsmeding on Sunday March 12, 2023 @04:10PM (#35125574)
While I get your point...

> It also makes it easier to add arguments to the end if re-running it.

I'd like to point out that a redirection doesn't have to be the last thing in a command. E.g. the common 'echo >&2 "some message"' works fine.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by alas44 on Sunday March 12, 2023 @05:38PM (#35126619)
I share the same opinion, I was made fun of with the "useless use of cat award" but find it so convienent to cat | grep then cat | grep | awk | wc then whatever with data flowing left to right and modifying the command sequence as I explore the file content.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by kccqzy on Sunday March 12, 2023 @05:50PM (#35126742)
Besides it's also a fail-safe way to make sure the program doesn't modify the original file.

    random-app ./myfile.txt 
The app opens the file on its own. It could decide to write to it.

    cat ./myfile.txt | random-app
The app receives chunks of the file from a pipe. It doesn't know where the original file is.

This is especially useful if you aren't sure the program will by default modify the file (such as formatters).

Re:Nushell.sh ls | where size > 10mb | sort-by modified by codetrotter on Sunday March 12, 2023 @06:27PM (#35127115)
Yeah but you get the same from

    random-app < ./myfile.txt
Re:Nushell.sh ls | where size > 10mb | sort-by modified by a1369209993 on Sunday March 12, 2023 @08:45PM (#35128955)
Only if random-app is well-behaved (and if you knew it was you could just use `random-app ./myfile.txt`).

  $ echo foo > a.txt
  $ <a.txt ( echo bar >/proc/self/fd/0 )
  $ cat a.txt
  bar
Re:Nushell.sh ls | where size > 10mb | sort-by modified by NoZebra120vClip on Sunday March 12, 2023 @09:33PM (#35129496)
That's bizarre and perverse! Surely someone has brought it up as a bug before that stdin is read-write! Is there some Posix standard preventing the standard shells from opening stdin as read-only???
Re:Nushell.sh ls | where size > 10mb | sort-by modified by a1369209993 on Sunday March 12, 2023 @10:49PM (#35130331)
stdin is readonly. `<a.txt ls /proc/self/fd/0` generally gives 'lr-x------' for the permissions. The problem is that having a open file descriptor to a file lets a program get and/or act-as-if-it-had a path to the file; /proc/self/fd/ is just the easiest way to do that.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by NoZebra120vClip on Monday March 13, 2023 @04:43AM (#35132839)
Ah, I believe I see what's happening there.

So, hiding the pathname from an untrusted tool via an unnamed pipe is a Posix security measure. Still bizarre!

Who was the guy who awarded UUOC prizes? Wasn't he a real dyed-in-the-wool security wonk?

Re:Nushell.sh ls | where size > 10mb | sort-by modified by NoZebra120vClip on Tuesday March 14, 2023 @01:37AM (#35147795)
So the thing is that this is a nothingburger. Just because you gave a pathname to an untrusted app means nothing. You've already trusted the app with your user account. It could already overwrite or vandalize that file no matter how you invoked it. Just because you indicate that file is special to you doesn't change anything in the threat model here. For all you know, the app could just traverse the entire directory tree and trash every file it could possibly write to, or just confine the damage to your $HOME.

There's no reason IMHO to avoid using a file as an argument, or directly as stdin. If you don't trust an app, don't run it in your user account; you run it in a sandbox, right? This is 2023.

Now a case could be made for defending against misbehavior by an app that might write to an fd by mistake, but as a1369209993 demonstrates, writing to stdin is a very deliberate choice, as you'll need to look up a pathname and deliberately open that file as writable. That's not misbehavior, that's malice, and that doesn't belong anywhere near your user account in the first place.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by scubbo on Monday March 13, 2023 @12:13AM (#35130965)
Quite. I had occasion to share this[0] in another thread, and it's relevant here, too:

> When I offer a pipeline as a solution I expect it to be reusable. It is quite likely that a pipeline would be added at the end of or spliced into another pipeline. In that case having a file argument to grep screws up reusability, and quite possibly do so silently without an error message if the file argument exists. I. e. `grep foo xyz | grep bar xyz | wc` will give you how many lines in xyz contain bar while you are expecting the number of lines that contain both foo and bar. Having to change arguments to a command in a pipeline before using it is prone to errors. Add to it the possibility of silent failures and it becomes a particularly insidious practice.

[0] https://stackoverflow.com/a/16619430/1040915

Re:Nushell.sh ls | where size > 10mb | sort-by modified by marcosdumay on Sunday March 12, 2023 @04:47PM (#35126053)
> Historically we would discourage shell users from...

For no good reason at all.

It's not less confusing, and when shell programmers start talking about how wasteful it is to fork another process, it's because all the good explanations were already refuted.

Also, it's worth pointing that this style was never unanimous. There is a very vocal minority that pushes for it, and a wide non-vocal majority that says basically "whatever, it's not like there's any difference" if you go and ask the question.

Re:Nushell.sh ls | where size > 10mb | sort-by modified by mort96 on Sunday March 12, 2023 @05:27PM (#35126517)
The use of cat is not useless. Having a "read this file" command be the source of the pipeline makes a lot of sense. It makes it seamless to replace the cat with a pv if you want progress, zcat if you're reading a gzipped file, curl if you want to use a URL, etc.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by codetrotter on Sunday March 12, 2023 @05:42PM (#35126661)
Fair point.

But how often do you make those kinds of changes in any of your scripts and not have to change anything? For me, exactly 0 times

Re:Nushell.sh ls | where size > 10mb | sort-by modified by mort96 on Sunday March 12, 2023 @05:53PM (#35126775)
Quite frequently? "Oh this operation turned out to take a while, I'll use pv instead of cat", or "oh what I grepped for wasn't in the current log, it's probably in the previous log that's gzipped, I'll replace 'cat current.log' with 'zcat previous.log'".
Re:Nushell.sh ls | where size > 10mb | sort-by modified by asda22sdasd on Sunday March 12, 2023 @06:13PM (#35126988)
Sounds like you describe a interactive shell session, not stuff that happens in that script that was written for a specific purpose.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by coldtea on Sunday March 12, 2023 @07:07PM (#35127782)
You'd be surprised
Re:Nushell.sh ls | where size > 10mb | sort-by modified by pseudalopex on Sunday March 12, 2023 @07:18PM (#35127958)
The 1st scenario and things similar to the 2nd happen in script development.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by version_five on Sunday March 12, 2023 @05:42PM (#35126668)
I'm also a big fan of doing it the wrong way. (While I understand there is no practical difference) I find it more natural, because it's symmetrical (or parallel:

  $ < file util1 | util2 | util3 | ...
The first util is the originator of the data pipeline, and it plus the file are the first command

  $ cat file | util1 | ... 
Each utility is it's own thing, being fed a data stream that came from cat. I know it's just personal preference but it feels neater
Re:Nushell.sh ls | where size > 10mb | sort-by modified by e12e on Sunday March 12, 2023 @05:53PM (#35126782)
Re:Nushell.sh ls | where size > 10mb | sort-by modified by pxc on Sunday March 12, 2023 @06:01PM (#35126865)
`save` seems to be a general serialization command, which lets you save Nu objects in formats like JSON, YAML, and others. The `--raw` option seems like a convenience for enabling that tee-like usage without relying on external programs, but the driving use case is serializing structured data rather than unstructured text or byte streams.
Re:Nushell.sh ls | where size > 10mb | sort-by modified by e12e on Sunday March 12, 2023 @09:11PM (#35129252)
Ah, I failed to realize that nushell deals with rich data.

I think this might hit the sweetspot for me, for dealing with json - I never could get a feel for "jq" like I have with sed/awk, and don't much enjoy powershell.

I don't think I need a new shell (bash is fine - and I would probably move to fish for a new shell if I were to change) - but I certainly need a sane way to wrangle json (hello docker inspect!) and yaml (hello kubectl/k8s and github actions!).

Re:Nushell.sh ls | where size > 10mb | sort-by modified by pxc on Monday March 13, 2023 @12:06AM (#35130911)
FWIW, I've made basically the same call for now: Fish has been my daily driver for some time, but I have a lot of interest in Nushell and some real affection for Elvish, and I think I'd like to eventually make the switch to one of these newfangled shells with the structured pipelines one day.

The great thing about shell languages is that you get to practice the scripting language with your everyday interactive usage, which is one of the main reasons I'd eventually like to switch, even if I see that perhaps the most exciting use cases are wrangling JSON and YAML in scripts.