News for hackers. Stuff that matters.
Re:The Beauty of Unix Pipelines by jcranmer on Thursday June 04, 2020 @05:15PM (#23421901)
> (1) everything is text

And lists are space-separated. Unless you want them to be newline-separated, or NUL-separated, which is controlled by an option that may or may not be present for the command you're invoking, and is spelled completely differently for each program. Or maybe you just quote spaces somehow, and good luck figuring out who is responsible for inserting quotes and who is responsible for removing them.

Re:The Beauty of Unix Pipelines by gorgoiler on Thursday June 04, 2020 @05:26PM (#23422023)
To criticize sh semantics without acknowledging that C was always there when you needed something serious is a bit short sighted.

There are two uses of the Unix “api”:

[A] Long lived tools for other people to use.

[B] Short lived tools one throws together oneself.

The fact that most things work most of the time is why the shell works so well for B, and why it is indeed a poor choice for the sort of stable tools designed for others to use, in A.

The ubiquity of the C APIs of course solved [A] use cases in the past, when it was unconscionable to operate a system without cc(1). It’s part of why they get first class treatment in the Unix man pages, as old fashioned as that seems nowadays.

Re:The Beauty of Unix Pipelines by jcranmer on Thursday June 04, 2020 @05:52PM (#23422347)
There's a certain irony in responding to criticism of that which you're extolling by saying not to use it.

And the only reason I might be pushed down that path is because the task I'm working on happens to involve filenames with spaces in them (without those spaces, the code would work fine!), because spaces are a reasonable thing to put in a filename unless you're on a Unix system.

Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @04:17AM (#23426209)
> because spaces are a reasonable thing to put in a filename unless you're on a Unix system.

Putting spaces on a filename is atrocious and should be disallowed by modern filesystems. It is like if you could put spaces inside variable names in Python. Ridiculous.

Re:The Beauty of Unix Pipelines by necrotic_comp on Friday June 05, 2020 @08:03AM (#23427518)
At Goldman, the internal language Slang has spaces in variable names. It's insane at first glance and I got into an argument with my manager about why in the world this was acceptable, and he could give me no other answer than "this is the way it is".

But when you realize that space isn't a valid separator between token, seeing things like "Class to Pull Info From Database::Extractor Tool" actually becomes much easier to read and the language becomes highly expressive, helped somewhat by the insane integration into the firm's systems.

I was on your side until I tried it, but it can actually be quite useful, esp. if everything is consistent.

Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @08:19AM (#23427627)
> I was on your side until I tried it, but it can actually be quite useful, esp. if everything is consistent.

On my side? You cannot imagine how extreme one can be... If I had my way, programming languages would only allow single-letter variables and space would be the multiplication operator.

Re:The Beauty of Unix Pipelines by nerdponx on Friday June 05, 2020 @09:44AM (#23428419)
What possible justification could you have for making this mandatory?

Math notation being dependent on single-character names is an artifact of "concatenation as multiplication" -- which in my opinion is a useful but sloppy historical convention with little merit or relevance in non-handwritten media.

Re:The Beauty of Unix Pipelines by necrotic_comp on Friday June 05, 2020 @09:46AM (#23428433)
Okay - I wasn't meaning to pick a fight here. Just saying that there are definite use cases where spaces in variable names can work and actually be useful. That's all.
Re:The Beauty of Unix Pipelines by HeadsUpHigh on Friday June 05, 2020 @06:44AM (#23427090)
I'm not going back to CamelCase or underscores for my normal day to day file naming. The problem with spaces only exists inside the IT world and it's something they should find a way around.
Re:The Beauty of Unix Pipelines by jyounker on Friday June 05, 2020 @12:54PM (#23430663)
At the point I'd be happy if all unix tools had a `--json` option. Then the whole space issue goes away. So do a bunch of parsing issues.

Unix tools standardized a character representation (e.g. everything is ascii). The next step is a unified standard the syntactical representation.

Re:The Beauty of Unix Pipelines by laumars on Friday June 05, 2020 @01:56PM (#23431611)
ASCII isn’t just a “character representation” though. It includes code points for data serialisation and transmission control. You can represent tables in ASCII without resorting to printable characters nor white space as field separators.

I don’t why POSIX tools don’t already do this but I suspect it’s because most tools would have evolved by accident rather than being designed from the ground up with edge cases in mind.

Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @07:16AM (#23427254)
I agree. The file load/save dialogs of all GUI should work in such a way that spaces typed by the users in a filename field are always transparently changed to something different (for example, the unicode non-breaking space).
Re:The Beauty of Unix Pipelines by skissane on Friday June 05, 2020 @08:05AM (#23427530)
> (for example, the unicode non-breaking space)

That's potentially even worse in a file name than plain old space is. It looks like a normal space, but isn't.

If you are going to replace spaces in file names with something else, underscores are the best option I think.

Re:The Beauty of Unix Pipelines by HeadsUpHigh on Friday June 05, 2020 @10:24AM (#23428905)
Sounds like taking the "I know better than the users to another level".
Re:The Beauty of Unix Pipelines by outadoc on Friday June 05, 2020 @04:26AM (#23426257)
You should take a stroll in the real world sometime, where spaces and Unicode exists :)
Re:The Beauty of Unix Pipelines by gorgoiler on Friday June 05, 2020 @01:04PM (#23430787)
The parent comment is extreme, and the real world is indeed very diverse, but I would also be quite surprised to find a software project with spaces in internal filenames.
Re:The Beauty of Unix Pipelines by outadoc on Thursday June 18, 2020 @04:32AM (#23561033)
So would I, but the world is, thankfully, not entirely composed of software projects.
Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @04:30AM (#23426280)
Yeah, any unicode character is OK on a filename, except maybe space and slash.
Re:The Beauty of Unix Pipelines by skissane on Friday June 05, 2020 @08:10AM (#23427563)
I don't think C0 or C1 controls should be allowed in filenames.

Why allow them? And they potentially pose security issues.

With some, admittedly somewhat obscure [1][2], terminal emulators, C0 and C1 controls can even be used to execute arbitrary code. You could say these terminal emulators are insecure, and you may well be right, but the fact is they exist.

[1] See for example page 211 of https://static.zumasys.com/zumasys/atfiles/manuals/at7/AccuT... which documents the ESC STX < command CR escape sequence which tells the AccuTerm terminal emulator to run an arbitrary command

[2] Also the APC C1 control can make Kermit run arbitrary commands, if you've executed "SET APC UNCHECKED" (which is not the default setting) – see http://www.kermitproject.org/onlinebooks/usingckermit3e.pdf page numbered 300 (page 310 of PDF)

Re:The Beauty of Unix Pipelines by nerdponx on Friday June 05, 2020 @09:46AM (#23428438)
I agree. Personally I think the Linux kernel should have a compilation-time option to disallow various special characters in filenames such as ASCII control characters and colons, so users who want to maintain a sane system can give themselves the option to do so.
Re:The Beauty of Unix Pipelines by AnIdiotOnTheNet on Friday June 05, 2020 @10:39AM (#23429069)
So um, how would you work with files that came from a foreign file system? Would the kernel just crash when it sees them? Would they be effectively untouchable, like filenames with a '/' encoded in them?
Re:The Beauty of Unix Pipelines by skissane on Friday June 05, 2020 @12:18PM (#23430224)
Previously, I proposed addressing this with a mount option and a filesystem volume flag [1]. The mount option would prevent creating new files/directories with "bad names" but would still allow accessing ones which already exist. The filesystem volume flag would make bad names on that volume be considered fsck errors. (If the kernel found such a bad name on a volume with the flag enabled, it should log that as a filesystem error, and either return an IO error, or just pretend the file doesn't exist – whatever Linux does now when it finds a file on disk whose name contains a null byte or forward slash.)

Using a mount option and/or filesystem volume flag means it works fine with legacy volumes which might contain those bad names. On such legacy volumes, either that volume flag is not supported by the filesystem, or else it is but is disabled for that particular volume. If you mount it with the mount option, you can still access bad names on that volume, you just can't create new ones; if you need to create new bad names, you just (re)mount it with the mount option disabled.

[1] https://news.ycombinator.com/item?id=22873153

Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @10:11AM (#23428743)
and spaces! ;)
Re:The Beauty of Unix Pipelines by nerdponx on Friday June 05, 2020 @09:41AM (#23428390)
I disagree with this immensely.

Also, some languages do allow whitespace in variable names like R and SQL, so long as the variable names are quoted or escaped properly.

Re:The Beauty of Unix Pipelines by tomaskafka on Friday June 05, 2020 @11:58AM (#23429977)
Are your kids named NameSurname? Did you meet EuropeanBrownBear on your last trip to the mountains? :)
Re:The Beauty of Unix Pipelines by coliveira on Thursday June 04, 2020 @06:03PM (#23422498)
The shell works with spaces, you just need to be careful to quote every file name. The advantage of using file names without spaces is that you can avoid the quotes.
Re:The Beauty of Unix Pipelines by viraptor on Thursday June 04, 2020 @08:01PM (#23423499)
Please don't ever assume you can avoid spaces and quotes. It's just a time bomb.

It's like people saying you don't need to escape SQL values because they come from constants. Yes, they do... today.

It's not just quoting either. It's setting the separator value and reverting it correctly. It's making sure you're still correct when you're in a function in an undefined state. It's a lot of overhead in larger projects.

Re:The Beauty of Unix Pipelines by CountSessine on Thursday June 04, 2020 @10:37PM (#23424459)
Sure, but in a larger project your won’t be coding in shell script, right?
Re:The Beauty of Unix Pipelines by viraptor on Thursday June 04, 2020 @11:35PM (#23424846)
Larger projects will often include small shell scripts for deployment / maintenance / various tasks.
Re:The Beauty of Unix Pipelines by coliveira on Friday June 05, 2020 @06:10PM (#23434613)
What I am saying is nothing new. This has always been the policy of UNIX developers. If you use only file names without space, then you can make it easier to use tools that depend on this. Of course, if you're writing applications in shell then you cannot rely just on convention.
Re:The Beauty of Unix Pipelines by tsimionescu on Friday June 05, 2020 @04:08AM (#23426162)
The shell (bash, anyway) has a weird mix of support and lack of support for spaces/lists/etc.

For example, there is no way to store the `a b "c d"` in a regular variable in a way where you can then call something similar to `ls $VAR` and get the equivalent of `ls a b "c d"`. You can either get the behavior of `ls a b c d` or `ls "a b c d"`, but if you need `ls a b "c d"` you must go for an array variable with new syntax. This isn't necessarily a big hurdle, but it indicates that the concepts are hard to grasp and possibly inconsistent.

Re:The Beauty of Unix Pipelines by account42 on Friday June 05, 2020 @06:15AM (#23426894)
Bash has arrays:

    var=( a b "c d" )
    ls "${var[@]}"
POSIX shell can get you what you want with eval:

    var='a b "c d"'
    eval "ls $var"
There is also one array variable in POSIX shells, the argument list:

    set -- a b "c d"
    ls "$@"
Re:The Beauty of Unix Pipelines by teddyh on Friday June 05, 2020 @04:37AM (#23426314)
That has nothing to do with spaces, but more to do with the rather lackluster support for arrays in most Bourne-style shells.
Re:The Beauty of Unix Pipelines by gorgoiler on Thursday June 04, 2020 @11:37PM (#23424853)
Indeed, and the shell lives and breathes spaces. Arrays of arguments are created every time one types a command. Without spaces-as-magic we’d be typing things like this all the time:

  $ exec(‘ls’, ‘-l’, ‘A B C’)
Maybe that’s unrealistic? I mean, if the shell was like that, it probably wouldn’t have exec semantics and would be more like this with direct function calls:

  $ ls(ls::LONG, ‘A B C’)
Maybe we would drop the parentheses though — they can be reasonably implied given the first token is an unspaced identifier:

  $ ls ls::LONG, ‘A B C’
And really, given that unquoted identifiers don’t have spaces, we don’t really need the commas either. Could also use ‘-‘ instead of ‘ls::’ to indicate that an identifier is to be interpreted locally in the specific context of the function we are calling, rather than as a generic argument.

  $ ls -LONG ‘A B C’
If arguments didn’t have spaces, you could make the quotes optional too.

QED

Re:The Beauty of Unix Pipelines by tsimionescu on Friday June 05, 2020 @04:13AM (#23426188)
I don't think this is the problem people usually complain about.

The much bigger problem is that spaces make text-only commands compose badly.

    $ ls -l `find ./ -name *abc*`
Is going to work very nicely if file names have certain properties (no spaces, no special chars), and it's going to break badly if they don't. Quoting is also very simple in simple cases, but explodes in complexity when you add variables and other commands into the mix.
Re:The Beauty of Unix Pipelines by hnlmorg on Friday June 05, 2020 @05:08AM (#23426491)
That's still just a shell problem since the shell is expanding `find` and placing the unquoted result into the command line. Some non-POSIX shells don't have this issue. For example with murex you could use either of these two syntaxes:

    ls -l ${find ./ name *abc*}
    # return the result as a single string

    ls -l @{find ./ name *abc*}
    # return the result as an array (like Bourn shell)
So in the first example, the command run would look something like:

    ls -l "foo abc bar"
whereas in the second it would behave more like Bourne shell:

    ls -l foo abc bar
As an aside, the example you provided also wouldn't work in Bourne shell because the asterisks would be expanded by the shell rather than `find`. So you'd need to quote that string:

    ls -l `find ./ -name "*abc*"`
This also isn't a problem in murex because you can specify whether to audo-expand globbing or not.

With POSIX shells you end up with quotation mark soup:

    ls -l "`find ./ -name '*abc*'`"
(and we're lucky in this example that we don't need to nest any of the same quotation marks. It often gets uglier than this!)

Murex also solves this problem by supporting open and close quote marks via S-Expression-style parentheses:

    ls -l (${find ./ -name (*abc*)})
Which is massively more convenient when you'd normally end up having to escape double quotes in POSIX shells.

---

Now I'm not trying to advocate murex as a Bash-killer, my point is that it's specifically the design of POSIX shells that cause these problems and not the way Unix pipelines work.

Re:The Beauty of Unix Pipelines by bregma on Friday June 05, 2020 @07:57AM (#23427477)
Ah, more problems with shell expansion and substitution.

Pipelines to the rescue. If you used a pipeline instead of shell expansion, it can deal with spaces just peachy.

    $ find . -name "*abc*" -print0 | xargs -0 ls -l
There's no argument that bash (and other shells) make dealing with spaces and quotes troublesome. That has nothing to do with pipelines.
Re:The Beauty of Unix Pipelines by tsimionescu on Friday June 05, 2020 @12:27PM (#23430322)
Does every unix utility suport -print0 or similar?

My point wasn't specifically about find, but about the problems of text and other unstructured input/output in general.

Update: reading a bit around the internet, it looks like it's not possible to do something like this :

    find / -print0 | \
        grep something | \
        xargs -0 cat
As grep will mangle the output from find -print0.
Re:The Beauty of Unix Pipelines by jcranmer on Friday June 05, 2020 @09:00AM (#23427972)
And note that, all the way in my original argument, I complained that every command used a different way of spelling that changed delimiter character.
Re:The Beauty of Unix Pipelines by thekaleb on Thursday June 04, 2020 @08:30PM (#23423720)
Some shells handle this quoting better than others.
Re:The Beauty of Unix Pipelines by kbenson on Thursday June 04, 2020 @09:32PM (#23424120)
That's when it's useful to use Perl as a substitute for a portion of or the entire pipeline.
Re:The Beauty of Unix Pipelines by de_watcher on Friday June 05, 2020 @01:53AM (#23425467)
Choosing space as a separator is a useful convention that saves you typing in additional code afterwards if you want to run a short command.

It's like there is a shortcut that some people use that you want to wall in because it doesn't look pretty to you.

Re:The Beauty of Unix Pipelines by gorgoiler on Thursday June 04, 2020 @11:49PM (#23424910)
The biggest pain comes from untarring some source code and trying to build it while in ~/My Tools Directory. Spaces in directories that scupper build code that isn’t expecting them is the fatal mixing of worlds.

In most other cases I’ve never really had a problem with “this is a place where spaces are ok” (e.g. notes, documents, photos) and “this is a place where they are not ok” — usually in parts of my filesystem where I’m developing code.

It’s fine to make simplifying assumptions if it’s your own code. Command history aside, most one liners we type at the shell are literally throwaways.

I think I was clear that they aren’t the only category of program one writes and that, traditionally on Unix systems, the counterpart to sh was C.

Re:The Beauty of Unix Pipelines by tsimionescu on Friday June 05, 2020 @04:16AM (#23426201)
The problem is that the pipeline model is extremely fragile and breaks in unexpected ways in unexpected places when hit with the real world.

The need to handle spaces and quotes can take you from a 20 character pipeline to a 10 line script, or a C program. That is not a good model whichever way you look at it.

Re:The Beauty of Unix Pipelines by mbreese on Friday June 05, 2020 @09:27AM (#23428221)
Pipelines are mainly for short-lived, one-off quick scripts. But they are also really useful for when you control the input data. For example, if you need a summary report based on the results of a sql query.

If you control the inputs and you need to support quotes, spaces, non-white space delimiters, etc... in shell script, then that’s on you.

If you don’t control the inputs, then shell scripts are generally a poor match. For example, if you need summary reports from a client, but they sometimes provide the table in xlxs or csv format — shell might not be a good idea.

Might be controversial, but I think you can tell who works with shell pipes the most by looking at who uses CSV vs tab-delimited text files. Tabs can still be a pain if you have spaces in data. But if you mix shell scripts with CSV, you’re just asking for trouble.

Re:The Beauty of Unix Pipelines by diegocg on Friday June 05, 2020 @07:37AM (#23427365)
That's not the fault of pipes, that's the fault of shells like bash. There are shells that deal with spaces and quotes perfectly fine.
Re:The Beauty of Unix Pipelines by necrotic_comp on Friday June 05, 2020 @07:59AM (#23427488)
I've been scripting stuff on the pipeline for over a decade and haven't really run into this much.

You can define a field separator in the command line with the environment variable IFS - i.e. 'IFS=$(echo -en "\n\b");' for newlines - which takes care of the basic cases like spaces in filenames/directory names when doing a for loop, and if I have other highly structured data that is heavily quoted or has some other sort of structure to it, then I either normalize it in some fashion or, as you suggest, write a perl script.

I haven't found it too much of a burden, even when dealing with exceptionally large files.

Re:The Beauty of Unix Pipelines by xkucf03 on Monday June 08, 2020 @04:33PM (#23460449)
I agree that spaces, quotes, encodings, line-ends etc. are problem, but this is not problem of „the pipeline model“ – this is problem of ambiguously-structured data. See https://relational-pipes.globalcode.info/v_0/classic-example...
Re:The Beauty of Unix Pipelines by nerdponx on Friday June 05, 2020 @09:39AM (#23428372)
Kind of / not really.

Also Zsh solves 99% of my "pipelines are annoying" and "shell scripts are annoying" problems.

Even on systems where I can't set my default shell to Zsh, I either use it anyway inside Tmux, or I just use it for scripting. I suppose I use Zsh the way most people use Perl.

Re:The Beauty of Unix Pipelines by nerdponx on Thursday June 04, 2020 @05:36PM (#23422165)
C was always there when you needed something serious

There is a world of stuff in between "I need relatively low-level memory management" and "I need a script to just glue some shit together".

For that we have Python and Perl and Ruby and Go, or even Rust.

Re:The Beauty of Unix Pipelines by gorgoiler on Friday June 05, 2020 @02:34AM (#23425665)
Yes! I love 2020!

(My point about C was in the historical context of Unix, which is relevant when talking about its design principles.)

Re:The Beauty of Unix Pipelines by 1MachineElf on Thursday June 04, 2020 @07:01PM (#23422990)
Going with the same theme, C itself was an innovation meant to fill the space between Assembler (in this context, "A") and B[0].

[0] https://en.wikipedia.org/wiki/B_(programming_language)

Re:The Beauty of Unix Pipelines by klyrs on Thursday June 04, 2020 @10:10PM (#23424310)
That isn't really supported by the article you linked -- it describes C as an extension of B multiple times.
Re:The Beauty of Unix Pipelines by 1MachineElf on Friday June 05, 2020 @10:35AM (#23429020)
I only linked that because most people don't know what B is, and I suppose the difference between "innovation meant to fill the space" and "extention" wasn't so important to me.

My source for the claim itself is from 14m30s of Brian Cantrill's talk "Is It Time to Rewrite the Opersting System in Rust" https://youtu.be/HgtRAbE1nBM

Re:The Beauty of Unix Pipelines by jacquesm on Friday June 05, 2020 @03:13AM (#23425870)
To be more precise: A->BCPL->B->C
Re:The Beauty of Unix Pipelines by viraptor on Thursday June 04, 2020 @07:04PM (#23423019)
This doesn't always hold in Linux though. Some proc entries are available only via text entries you have to parse whether that's in sh or C. There's simply no public structured interface.
Re:The Beauty of Unix Pipelines by JdeBP on Friday June 05, 2020 @03:12AM (#23425867)
This is one of the superior aspects of FreeBSD: no parsing of human-readable strings to get back machine-readable information about the process table. It's available directly in machine-readable form via sysctl().
Re:The Beauty of Unix Pipelines by pwdisswordfish2 on Thursday June 04, 2020 @08:38PM (#23423770)
[C] Long lived tools one throws together oneself

Stable tools designed for oneself

Re:The Beauty of Unix Pipelines by Koshkin on Thursday June 04, 2020 @08:55PM (#23423891)
The unknown knowns?
Re:The Beauty of Unix Pipelines by hnlmorg on Thursday June 04, 2020 @05:22PM (#23421983)
That's a POSIX shell thing rather than a Unix pipeline thing. Some non-POSIX shells don't have this problem while still passing data long Unix pipes

Source: I wrote a shell and it solves a great many of the space / quoting problems with POSIX shells.

Re:The Beauty of Unix Pipelines by buzzkillington on Thursday June 04, 2020 @09:47PM (#23424191)
Same, I've built toy shells [0] that used xml, s-expressions and ascii delimited text [1]. The last was closest to what unix pipes should be like. Of course it breaks ALL posix tools, but it felt like a shell that finally works as you'd expect it to work.

[0] Really just input + exec + pipes.

[1] https://en.wikipedia.org/wiki/Delimiter#ASCII_delimited_text

Re:The Beauty of Unix Pipelines by hnlmorg on Friday June 05, 2020 @04:45AM (#23426354)
As long as you re-serialise that data when piping to coreutils (etc) you shouldn't have an issue.

This is what my shell (https://github.com/lmorg/murex) does. It defaults to using JSON as a serialisation format (that's how arrays, maps, etc are stored, how spaces are escaped, etc) but it can re-serialise that data when piping into executables which aren't JSON aware.

Other serialisation formats are also supported such as YAML, CSV and S-Expressions too. I had also considered adding support for ASCII records but it's not a use case I personally run into (unlike JSON, YAML, CSV, etc) so haven't written a marshaller to support it yet.

Re:The Beauty of Unix Pipelines by buzzkillington on Friday June 05, 2020 @09:15AM (#23428115)
Ascii records have worked better than json, xml, s-expressions (better as in provide the same bang for a lot less buck) in every case I've used them for.

I have no idea why I am the only person I know who uses them regularly.

Re:The Beauty of Unix Pipelines by hnlmorg on Sunday June 07, 2020 @04:19AM (#23445714)
ASCII records have their own limitations though:

- They can be as easily edited by hand like other serialisation formats which use printable characters as their deliminators

- It's not clearly defined how you'd use them for non-tabulated data (such as JSON, XML, S-Expressions)

- There isn't any standard for escaping control characters

- They're harder to differentiate between unserialised binary formats

- They can't be used as a primitive like JSON is to Javascript and S-Expressions is to Lisp.

And if we're both honest, reducing the serialisation overhead doesn't gain you anything when working in the command line. It's a hard enough sell getting websites to support BSON and at least there, there is a tangible benefit of scale.

Not that I'm dismissing ASCII records, they would have been better than the whitespace mess we currently have in POSIX shells. However I don't agree ASCII records are better that current JSON nor S-Expressions.

Re:The Beauty of Unix Pipelines by JdeBP on Friday June 05, 2020 @03:14AM (#23425878)
Here's one tool that it does not break. (-:

* http://jdebp.uk./Softwares/nosh/guide/commands/console-flat-...

Re:The Beauty of Unix Pipelines by gricardo99 on Thursday June 04, 2020 @07:24PM (#23423206)
>good luck figuring out ...

luck, or simply execute the steps you want to check by typing them on the command line. I find the pipes approach incredibly powerful and simple because you can compose and check each step, and assemble. That's really the point, and the power, of a simple and extensible approach.

Re:The Beauty of Unix Pipelines by 6gvONxR4sf7o on Thursday June 04, 2020 @05:19PM (#23421936)
God help you if your paths have spaces in them.
Re:The Beauty of Unix Pipelines by mfontani on Thursday June 04, 2020 @07:57PM (#23423473)
No need to invoke a deity to do that...

  echo 'Hi!' | tee "$(printf "foo\nbar\tqu ux.txt")"
  cat foo$'\n'bar$'\t'qu' 'ux.txt
IIRC the latter only works in ksh93+, but the former should be fine pretty much in any shell.
Re:The Beauty of Unix Pipelines by JdeBP on Friday June 05, 2020 @03:19AM (#23425897)
... except if one uses \c by mistake. \c has unspecified behaviour, and is one of the things that varies widely from one implementation of printf to another.

* https://unix.stackexchange.com/a/558665/5132

Re:The Beauty of Unix Pipelines by gorgoiler on Thursday June 04, 2020 @05:31PM (#23422091)
By “God” you may mean “libc”...

...if it was the 1970s. Nowadays we use Python etc for tools where it’s less acceptable for them to fall apart, though everything falls apart at some point, spaces or otherwise.

[dead]
Re:The Beauty of Unix Pipelines by bananabreakfast on Thursday June 04, 2020 @08:18PM (#23423635)
Don't you mean Raku? lol
Re:The Beauty of Unix Pipelines by naniwaduni on Thursday June 04, 2020 @09:12PM (#23423992)
Perl 5 was more suitable for these particular use cases than perl 6 ever tried to be.
Re:The Beauty of Unix Pipelines by anthk on Friday June 05, 2020 @08:04AM (#23427527)
Perl5 was and is amazing. Check the articles from https://linuxfocus.com and the old Linux Gazzete magazines from TLDP: https://linuxgazette.net/archives.html.
Re:The Beauty of Unix Pipelines by bityard on Thursday June 04, 2020 @05:43PM (#23422233)
Just quote paths? shrug
Re:The Beauty of Unix Pipelines by mehrdadn on Thursday June 04, 2020 @06:18PM (#23422648)
Try doing that in e.g. Make
Re:The Beauty of Unix Pipelines by blacksqr on Thursday June 04, 2020 @07:33PM (#23423277)
In the early 2k's I discovered that it was impossible to quote or escape spaces in filepaths in a makefile. I naively reported it to the Make maintainer as a bug.

The maintainer responded by saying, basically, that the behavior was deeply baked in and was not going to change.

Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @12:44AM (#23425156)
This is a nice feature of make. People who put spaces on filenames are justly punished by not being given the goodness of make.
Re:The Beauty of Unix Pipelines by mehrdadn on Friday June 05, 2020 @01:14AM (#23425301)
I can't tell if you're being sarcastic but this can result in innocent users who don't even know this fact to suddenly overwrite or wipe their files when trying to build someone's project.
Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @02:08AM (#23425537)
Nobody is innocent if they have filenames with spaces in their systems! /s
Re:The Beauty of Unix Pipelines by fao_ on Thursday June 04, 2020 @07:13PM (#23423090)
Try finding out which registry key you need to edit to get something to happen.

Some things across different systems have pitfalls as a result of their implementation, or some features have pitfalls. There are half a dozen things I can point to on windows that are worse than the "spaces in filenames". Hell, windows doesn't even support ISO 8601 dates, or times to be properly placed in files. It restricts symbols much more than Linux / Unix

Re:The Beauty of Unix Pipelines by yzmtf2008 on Thursday June 04, 2020 @09:35PM (#23424135)
who said anything about windows?
Re:The Beauty of Unix Pipelines by fao_ on Friday June 05, 2020 @07:01PM (#23434975)
The parent thread referenced windows as something that handles spaces in filenames. I'm not sure why I'm getting downvoted when my comment was relevant to the debate occurring, and my point (that all features have downfalls caused by the necessity of implementation within the surrounding system, and in the case of command line arguments, linguistics and common keyboards themselves) is something that according to the guidelines should be debated against, not downvoted.
Re:The Beauty of Unix Pipelines by yzmtf2008 on Friday June 05, 2020 @07:53PM (#23435332)
None of the parents of your comment mentioned Windows. Perhaps you have the wrong thread?
Re:The Beauty of Unix Pipelines by jlg23 on Friday June 05, 2020 @03:32AM (#23425964)
Or you just pipe through the program of your choice for which you know the syntax:

  echo "foo bar fnord" | cut --delimiter " " --output-delimiter ":" -f1-

  # foo:bar:fnord
Re:The Beauty of Unix Pipelines by bloopernova on Friday June 05, 2020 @09:35AM (#23428316)
I love learning new things about commands I've used for years.

    --output-delimiter ":"
Very cool!

However it does require the GNU version of cut, not the Mac OS X supplied BSD version. In zsh at least, you can do:

    alias cut='/usr/local/bin/gcut'
Re:The Beauty of Unix Pipelines by fomine3 on Thursday June 04, 2020 @10:01PM (#23424260)
The fact that filename can contain anything except NULL/slash is really pain. I often write a shell script that treats LF as separator but I know it's not good.
Re:The Beauty of Unix Pipelines by brokenmachine on Wednesday June 10, 2020 @08:38PM (#23483917)
Why not use null as a separator?

I agree it's messy but not that hard. Just set IFS to null.

Re:The Beauty of Unix Pipelines by enriquto on Friday June 05, 2020 @12:22AM (#23425058)
filenames are variable names. Choosing good filenames is the first step in writing a nice shell script in a situation that you control.
Re:The Beauty of Unix Pipelines by fomine3 on Friday June 05, 2020 @01:47AM (#23425439)
It's sometimes in control or out of control. Writing script in defensive way is better for security if filename is out of control.
Re:The Beauty of Unix Pipelines by Quekid5 on Friday June 05, 2020 @01:43AM (#23425419)
No, they aren't -- they are data.
Re:The Beauty of Unix Pipelines by ComputerGuru on Friday June 05, 2020 @12:03PM (#23430041)
I think this was a failure on behalf of early terminal emulator developers. End-of-field and end-of-record should have been supported by the tty (either as visual symbols or via some sort of physical distancing, etc even if portrayed just as a space and a new line respectively) so that the semantic definition could have been preserved/distinguished.
Re:The Beauty of Unix Pipelines by junon on Friday June 05, 2020 @12:17PM (#23430215)
TTYs in general are some of the most outdated and archiac pieces of technology still used today. They need a serious overhaul, but the problem is, any major change will break pretty much all programs.
Re:The Beauty of Unix Pipelines by chooseaname on Friday June 05, 2020 @07:48AM (#23427421)
All of which are so much easier to deal with than some binary format you have no spec on.

XML? SMH.

Re:The Beauty of Unix Pipelines by michaelcampbell on Friday June 05, 2020 @08:11AM (#23427570)
Ah yes, perfect is the enemy of the good.
Re:The Beauty of Unix Pipelines by pwdisswordfish2 on Friday June 05, 2020 @10:21AM (#23428877)
Works for me.
Re:The Beauty of Unix Pipelines by derefr on Thursday June 04, 2020 @07:41PM (#23423348)
People struggle so much with this, but I don't see what the point is at all. The fundamental problem that playing with delimiters solves, is passing arbitrary strings through as single tokens.

Well, that's easy. Don't escape or quote the strings; encode them. Turn them into opaque tokens, and then do Unix things to the opaque tokens, before finally decoding them back to being strings.

There's a reason od(1) is in coreutils. It's a Unix fundamental when working with arbitrary data. Hex and base64 are your friends (and Unix tools are happy to deal with both.)

Re:The Beauty of Unix Pipelines by brokenmachine on Friday June 05, 2020 @02:31AM (#23425648)
Never heard about this, would this affect performance?

Can you give some examples?