News for hackers. Stuff that matters.
Re:macOS command-line tools you might not know about by omginternets on Tuesday June 27, 2023 @09:16AM (#36492265)
If I had a nickel for each `cat foo.json | jq | pbcopy`, I'd be a rich man :)
Re:macOS command-line tools you might not know about by maleldil on Tuesday June 27, 2023 @09:58AM (#36492862)
That's a useless use of cat. You can use `jq . foo.json | pbcopy` or `jq < foo.json | pbcopy`.
Re:macOS command-line tools you might not know about by jdbartee on Tuesday June 27, 2023 @10:05AM (#36492960)
Speaking for myself, the first form is more natural- even if it’s a useless cat, because I’m always cat-ing files to see their structure. Then progressively tacking on different transforms. And then finally putting it in whatever I want as output.

It’s so ingrained, I’m more likely than not to just write it out that way even when I know exactly what I’m doing from the onset.

Re:macOS command-line tools you might not know about by jonnycomputer on Tuesday June 27, 2023 @10:37AM (#36493487)
Yes, this iterative procedure is often why "useless" cats get put into it. It's a very effective way of processing regular text information.

e.g.

I need to grab some info from textfile.txt to use as arguments to a function.

cat textfile.txt

looks like its comma delimited.

cat textfile.txt | cut -d, -f 2-5

ah, its the third and fourth column i need

cat textfile.txt | cut -d, -f 3-4 | grep '123456'

perfect

cat textfile.txt | cut -d, -f 3-4 | grep 123456 | tr , ' '

myfunc $(cat textfile.txt | cut -d, -f 3-4 | grep 123456 | tr , ' ')

Re:macOS command-line tools you might not know about by gumby on Tuesday June 27, 2023 @11:45AM (#36494503)
> cat textfile.txt

> looks like its comma delimited.

Interesting; why wouldn't you use `head`? Who knows how big textfile.txt is?

Re:macOS command-line tools you might not know about by jonnycomputer on Tuesday June 27, 2023 @12:38PM (#36495260)
generally, speaking, if you don't have an idea of how big the file is, or it would take up too much real-estate on your terminal window, sure. 100%. It was just an example.

lot's of times we sort of know what we are working with, but don't remember the particulars especially

Re:macOS command-line tools you might not know about by yrro on Tuesday June 27, 2023 @12:16PM (#36494913)
Don't forget to pipe head into 'cat -v'... that text file could contain _anything_!
Re:macOS command-line tools you might not know about by gunapologist99 on Tuesday June 27, 2023 @01:11PM (#36495824)
Thank you for pointing this out! This is much safer.
Re:macOS command-line tools you might not know about by lelandbatey on Tuesday June 27, 2023 @01:32PM (#36496116)
I really recommend folks use "less" over cat, especially keyboard oriented folks. Different terminal emulators don't always have the scroll behavior I want, not do they always allow me to search the file I'm looking at. "less" does all those things, in nearly every environment no matter the terminal emulator, and has other wonderful options to boot (chop long lines so they don't wrap can be nice for logs, line numbers can be VITAL, etc).

I still uselessly use cat though, it's such a nice way to build a pipeline.

Re:macOS command-line tools you might not know about by jmhammond on Tuesday June 27, 2023 @04:42PM (#36498355)
My useless cat is that I always use `cat file | less` when I could just `less file`.

I've been typing cat for over 25 years. Old habits die hard.

Re:macOS command-line tools you might not know about by rconti on Tuesday June 27, 2023 @07:07PM (#36499959)
I hate that when I use `less`, then quit, the output goes away.
Re:macOS command-line tools you might not know about by LgWoodenBadger on Wednesday June 28, 2023 @12:31PM (#36508860)
You can run "less -X" for that, but it may have other problems depending on how you use less (e.g. scrolling up, etc.)
Re:macOS command-line tools you might not know about by rovr138 on Tuesday June 27, 2023 @12:13PM (#36494862)
`file` will tell you too
Re:macOS command-line tools you might not know about by jonnycomputer on Tuesday June 27, 2023 @12:16PM (#36494911)
Won't tell you the delimiter.
Re:macOS command-line tools you might not know about by paulddraper on Tuesday June 27, 2023 @10:29AM (#36493350)
You could consider

    < foo.json jq | pbcopy
Re:macOS command-line tools you might not know about by jamespullar on Tuesday June 27, 2023 @11:35AM (#36494377)
I've been using bat as a cat replacement for a while now. It includes paging, syntax highlighting, line numbers, and is generally very performant.

https://github.com/sharkdp/bat

Re:macOS command-line tools you might not know about by patrec on Tuesday June 27, 2023 @01:42PM (#36496238)
If you're using zsh, you can just replace any instance of

    $ cat somefile ...
with

    $ <somefile ...
For bash, this only works if you have at least one `|`.
Re:macOS command-line tools you might not know about by ddingus on Tuesday June 27, 2023 @11:54PM (#36502079)
I did this last time I saw it come up and was surprised! Doing it makes perfect sense in hindsight. Neato!
Re:macOS command-line tools you might not know about by fastaguy88 on Tuesday June 27, 2023 @04:03PM (#36497947)
As a scientist who cares about reproducibility, the big difference between the "useless cat" and providing the input file name on the command line is that, in the latter case, the program can capture that file name and reproduce it. That is harder when using stdin.

Many of my programs and scripts start output with the line: # cmd arg1 arg2 arg3 ...

and simply echo back lines that start with '#'. That way, I have an internal record of the program that was run and the data file that was read (as well as previous parts of the analysis chain).

And, 'R' ignores lines starting with '#', so the record is there, but does not affect later analyses.

Re:macOS command-line tools you might not know about by nicky0 on Tuesday June 27, 2023 @10:17AM (#36493144)
In what way do you see those alternatives as superior?
Re:macOS command-line tools you might not know about by paulddraper on Tuesday June 27, 2023 @10:30AM (#36493372)
They avoid an unnecessary invocation of the cat executable.

Instead, they open a file descriptor and pass that.

Tiny difference but there you go.

Re:macOS command-line tools you might not know about by latexr on Tuesday June 27, 2023 @10:37AM (#36493489)
To add, searching for “useless use of cat” will yield several results for those interested in learning more. Other examples include “useless use of echo” and “useless use of ls *”.
Re:macOS command-line tools you might not know about by vram22 on Tuesday June 27, 2023 @07:30PM (#36500160)
Yes.

Even the Wikipedia page on cat has a section about that, titled eponymously.

https://en.m.wikipedia.org/wiki/Cat_(Unix)

Re:macOS command-line tools you might not know about by adrianmonk on Tuesday June 27, 2023 @11:04AM (#36493906)
Not just that, but also all the bytes have to go through an extra pipe. Presumably they're copied an extra time because of this.

When you run "cmd < file", the command reads from stdin, which pulls directly from the file. When you do "cat file | cmd", "cat" opens the file, reads from there, and writes to a pipe. Then "cmd" reads from its stdin, which is a pipe.

Re:macOS command-line tools you might not know about by chlorion on Tuesday June 27, 2023 @08:56PM (#36500885)
GNU cat will use the copy_file_range syscall when possible!

copy_file_range allows a user land program to copy data between two files without doing any user space work. Instead of reading data into a buffer and writing it back out to the destination, the kernel will somehow manage to move the data for you.

I think this will prevent any extra copies from occurring in situations where it can be used.

https://man.archlinux.org/man/copy_file_range.2

https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat...

Re:macOS command-line tools you might not know about by omginternets on Tuesday June 27, 2023 @11:53AM (#36494613)
>They avoid an unnecessary invocation of the cat executable.

And ... ?

Re:macOS command-line tools you might not know about by wpm on Tuesday June 27, 2023 @12:07PM (#36494783)
I teach shell scripting. Cat invocations are cheap and help learners understand and keep clear where input is coming from, and where it is going. There are no awards or benefits to reducing the number of lines, commands invoked, or finding the shortest possible way to perform a task in a script. There are plenty of detriments to reading and understanding though when we try to obfuscate this to save 1ms of execution time on a script that is going to execute near instantaneously anyways.

In short, I straight up don't care.

Re:macOS command-line tools you might not know about by revscat on Tuesday June 27, 2023 @12:21PM (#36494997)
I 100% agree with you. My only defense of OP is that `<` is something tends to be forgotten. Like everyone else in this thread I go to `cat` first for things like this. But sometimes I forget that even `<` exists, and the callout is a nice reminder.
Re:macOS command-line tools you might not know about by derefr on Tuesday June 27, 2023 @12:17PM (#36494926)
If the command is meant to stream through something really fast by using a large buffer size, then prepending a cat(1) will limit the incoming buffer size to ~4k.
Re:macOS command-line tools you might not know about by vram22 on Tuesday June 27, 2023 @07:19PM (#36500068)
Interesting.

Maybe use dd with one of its blocksize options, then?

Not at a terminal, can't check.

Re:macOS command-line tools you might not know about by gdavisson on Tuesday June 27, 2023 @03:50PM (#36497774)
It usually doesn't matter much, but there are some situations where it can matter a lot. For one thing, you can't use seek() on a pipe, so e.g. `cat bigfile | tail` has to read through the entire file to find the end, but `tail bigfile` will read the file backward from the end, completely skipping the irrelevant beginning and middle. With `pv bigfile | whatever`, pv (which is basically a pipeline progress indicator) can tell how big file is and tell you how for through you are as a percentage; with `cat bigfile | pv | whatever`, it has no idea (unless you add a flag to tell it). Also, `cat bigfile | head` will end up killing cat with a SIGPIPE signal after head exits; if you're using something like "Unofficial bash strict mode" [1], this will cause your script to exit prematurely.

Another sometimes-important difference is that if there are multiple input files, `somecommand file1 file2 file3` can tell what data is coming from which file; with `cat file1 file2 file3 | somecommand` they're all mashed together, and the program has no idea what's coming from where.

In general, though, I think it's mostly a matter of people's expertise level in using the shell. If you're a beginner, it makes sense to learn one very general way to do things (`cat |`), and use it everywhere. But as you gain expertise, you learn other ways of doing it, and will choose the best method for each specific situation. While `cat |` is usually an ok method to read from a file, it's almost never the best method, so expert shell users will almost never use it.

[1] http://redsymbol.net/articles/unofficial-bash-strict-mode/

Re:macOS command-line tools you might not know about by nojs on Tuesday June 27, 2023 @11:50AM (#36494558)
The “useless cat” meme needs to die. Everyone is aware that most commands accept a file argument, but looking up the arguments and their ordering is annoying and using cat for things like this is just fine.
Re:macOS command-line tools you might not know about by epcoa on Tuesday June 27, 2023 @12:15PM (#36494899)
The redirect always works though - that is not a program argument, that is handled by the shell. Apparently not everyone is aware of that.
Re:macOS command-line tools you might not know about by omginternets on Tuesday June 27, 2023 @11:52AM (#36494590)
This. "Useless cat" is more useful than "useless file-arg".
Re:macOS command-line tools you might not know about by hdb2 on Tuesday June 27, 2023 @01:30PM (#36496086)
granted, it is a little snarky and maybe the snark isn't appropriate in today's tech environment. but no, things like "useless use of cat" do not need to go away, because they make me better at what I do in little ways. those little ways add up over time.

> but looking up the arguments and their ordering is annoying

you seem to be arguing for complacency. taking your idea to an extreme, why learn to do _anything_ well?

Re:macOS command-line tools you might not know about by burnished on Tuesday June 27, 2023 @01:55PM (#36496387)
Everyone is not aware, new people are joining all the time.
Re:macOS command-line tools you might not know about by Someone on Tuesday June 27, 2023 @12:28PM (#36495101)
Is there any shell that has cat as a built-in?

Such a shell could remove some of the more common cases.

Re:macOS command-line tools you might not know about by hnlmorg on Tuesday June 27, 2023 @02:51PM (#36497117)
All of them do. Including bash. It’s just not the same syntax (ie ‘< filename’).

But I honestly think people who try to optimise away ‘cat’ are optimising the wrong thing. If one extra fork() is that detrimental then don’t use a shell scripting language.

For a lot of people, “useless” ‘cat’ enables them to write a pipeline in the order that their brain farts out the requirements for the pipeline. So they’ve optimised for human productivity. And given the human brain is slower than a few extra fork()s, I think optimising for one’s brain makes more sense here.

Re:macOS command-line tools you might not know about by Someone on Tuesday June 27, 2023 @05:04PM (#36498644)
Re:macOS command-line tools you might not know about by hnlmorg on Tuesday June 27, 2023 @05:57PM (#36499220)
Literally the next sentence after the one you quoted explains my point:

> It’s just not the same syntax (ie ‘< filename’).

Reading from a file isn’t a hard problem. Having a good UX for doing that is where most shells fall apart. And that’s basically what ‘cat’ offers here: an improved UX.

Having ‘cat’ as a shell builtin wouldn’t really solve the complaints raised by “useless use of” anyway because you’d still be piping (and in some cases, fork()ing too). You couldnt really use ‘cat’ as syntactic sugar for ‘<‘ because things start to get really weird if you want to pass flags to ‘cat’ or even redirect the output to something other than a pipe. And given ‘cat’ is POSIX (https://en.m.wikipedia.org/wiki/List_of_Unix_commands#/media...) the current behaviour of shells is, in my opinion, correct. This is why my own shell has a differently named builtin that approximately serves the purpose of ‘cat’ but for instances when you need the command built into the shell and it can’t just be passing a file handle to the next command (in my case, because i wanted to pass metadata out-of-band as well as the file contents)

Re:macOS command-line tools you might not know about by cratermoon on Tuesday June 27, 2023 @01:16PM (#36495918)