News for hackers. Stuff that matters.
Re:CLI: Improved by barbegal on Thursday August 30, 2018 @04:32PM (#17880477)
I feel like these tools very much go against the Unix philosophy of "Write programs that do one thing and do it well". They try to do the pretty user interface and the underlying operation in a single tool.

I prefer PowerShell in this respect where the output of each command is not text streams (as in the Unix world) but objects which can be operated on in a more object oriented way. You spend less time thinking about text parsing and more time thinking about the data you're working with.

I think the command line is great as a way of manipulating data streams but it is incredibly lacking as a user interface. There is very little consistency of the interface between commands and new commands and options aren't easily discoverable.

Re:CLI: Improved by l0b0 on Thursday August 30, 2018 @04:56PM (#17880675)
"One thing" has never been very well defined - basically every common Linux shell tool could be said to do more than one thing. GNU `grep` supports four different pattern styles (fixed strings, basic regex, extended regex and Perl regex), has lots of options for output formatting (counting, line numbers, whether to display file names, etc.) and has a bunch of rarely-used (but useful!) options for various corner cases, such as line buffering. Even GNU `cat`, the canonical "do one thing" tool, has several formatting options. If `grep` returned match objects rather than a semantically void binary stream it would be really easy to hand over formatting to another tool, but that just isn't how *nix tools work yet.
Re:CLI: Improved by Sean1708 on Friday August 31, 2018 @08:31AM (#17884957)
> Even GNU `cat`, the canonical "do one thing" tool, has several formatting options.

Huh, TIL. It's just never occurred to me to even bother checking the help for cat.

  $ cat --help
  Usage: cat [OPTION]... [FILE]...
  Concatenate FILE(s) to standard output.
  
  With no FILE, or when FILE is -, read standard input.
  
    -A, --show-all           equivalent to -vET
    -b, --number-nonblank    number nonempty output lines, overrides -n
    -e                       equivalent to -vE
    -E, --show-ends          display $ at end of each line
    -n, --number             number all output lines
    -s, --squeeze-blank      suppress repeated empty output lines
    -t                       equivalent to -vT
    -T, --show-tabs          display TAB characters as ^I
    -u                       (ignored)
    -v, --show-nonprinting   use ^ and M- notation, except for LFD and TAB
        --help     display this help and exit
        --version  output version information and exit
  
  Examples:
    cat f - g  Output f's contents, then standard input, then g's contents.
    cat        Copy standard input to standard output.
  
  GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
  Full documentation at: <http://www.gnu.org/software/coreutils/cat>
  or available locally via: info '(coreutils) cat invocation'
Re:CLI: Improved by bradknowles on Thursday August 30, 2018 @06:02PM (#17881204)
The thing I would like to see with grep is to have it optionally give me a 0 return code if it doesn't find anything.

I know why the authors chose to use a non-zero return code as the default, but when I'm using grep deep in a pipeline and doing my own checking of the results to see if nothing was found, I don't need grep bombing out the whole pipeline with a non-zero return code.

The alternative of being forced to use "(grep pattern||true)" instead of a plain "grep -q" is kinda painful.

Re:CLI: Improved by vitorbaptistaa on Thursday August 30, 2018 @10:08PM (#17882378)
I'm not in my computer now, but doesn't "grep -v" solves your issue? If I remember correctly, it inverts the query, so it would return 0 if there wasn't a match.
Re:CLI: Improved by Sean1708 on Friday August 31, 2018 @08:40AM (#17885022)
No, -v will return 0 unless every line matches:

  $ cat > foo
  here's
  some
  random
  text
  $ grep -v some foo; echo $?
  here's
  random
  text
  0
  $ grep -v bar foo; echo $?
  here's
  some
  random
  text
  0
  $ grep -vE "(here's|some|random|text)" foo; echo $?
  1
Re:CLI: Improved by Pete_D on Friday August 31, 2018 @01:40PM (#17887463)
How about "awk /pattern/"?
Re:CLI: Improved by mangamadaiyan on Friday August 31, 2018 @03:07AM (#17883530)
Would grep -c work for your use-case?
Re:CLI: Improved by burntsushi on Thursday August 30, 2018 @04:41PM (#17880540)
ripgrep definitely goes against the Unix philosophy. This is intentional. The Unix philosophy is a means to an end, and not an end unto itself. The key way that ripgrep violates the Unix philosophy is that it couples the filtering of what to search with the act of searching. You hit the nail on the head with styling, because the output of ripgrep is itself the thing that prevents composability.

However, from my own observations (and my own use), folks seem to appreciate having some amount of coupling and integration here. I've often seen folks claim that a legitimate use case (for them) for ripgrep is to "replace crazy 'find ./ ... | xargs' concoctions." The degree to which the aforementioned is "crazy" or not varies based on the individual, but there's a not insubstantial number of people who appreciate the succinctness that coupling brings.

As the maintainer, the coupling is annoying, because it means ripgrep needs to implement more stuff. For example, ripgrep provides a `--sort-files` option, which is something that standard grep doesn't need because you can fairly easy compose its output with 'sort'. You could do the same with ripgrep (ripgrep supports the standard composable grep output format), but then you lose the "pretty styling" that users appreciate. So you have no choice.

In terms of giving more structure to output, I am mostly unconvinced by your argument, but I've been happily using text streams as my user interface for a very long time, and I like its balance.

With that said, the next release of ripgrep will come with a --json flag, which enables structured output. :-)

Re:CLI: Improved by jgtrosh on Thursday August 30, 2018 @06:47PM (#17881479)
I wonder if there is or could be a good standard way of providing decoupled tools and their coupled interface.

Last year I wrote a friendly script to search content on YouTube, which prints human-readable content if plugged to a terminal and URLs otherwise, and another one which applies a pattern to turn such YouTube URLs to the underlying video or audio streams using youtube-dl. (I don't think this plays along nicely with YouTube's ToS but whatever I'm the only user.)

The obvious use case is to look at the output of the first command, and then pipe it to the second after choosing what to watch, and find a way to feed that to mplayer. So you'd want to provide a shortcut script, maybe make it interactive… but when does it go from an alias to a new program altogether? This interaction is very intuitive in the browser and I find trying to reproduce it with a CLI tool is quite challenging.

Btw I love ripgrep, it made me look very hip in front of colleagues a couple of times.

Re:CLI: Improved by bradknowles on Thursday August 30, 2018 @06:04PM (#17881217)
Which I can then pipe to "gron" so that I can make something sensible and greppable out of that json output, right?

;)

Re:CLI: Improved by dflock on Thursday August 30, 2018 @08:39PM (#17882064)
Well, you could pipe it to "jq" - https://stedolan.github.io/jq/
Re:CLI: Improved by needlepont on Friday August 31, 2018 @12:18PM (#17886764)
Which is a pretty abominable tool TBH. It breaks on just about every little edge and isn't very useful unless you like to see pretty colors and nested outputs + learn it's terrible output formatting spec. When using json I still use jansson for quick one offs.
Re:CLI: Improved by needlepont on Friday August 31, 2018 @08:00AM (#17884788)
I've never understood complaints like this. If you are proficient in the unix environment any sort of gymnastics can be handled via generation of some 'object' from plaintext and command generation on the fly. find $PATH -name '.c' -exec grep -l socket {} \; | awk ' {printf "mv %s %s\n",$0,sprintf("%s.old",$0)}' find $PATH -name '.c' -exec grep -l socket {} \; | awk ' BEGIN {n=0} {printf "{\"items\": \%s",sprintf("[\"%d\",\"%s\"]}\n",n++,$0)}'

If you aren't proficient or have an aesthetic or religious aversion to unix userland and traditional tools you'll play some other game I guess. Reinventing the wheel without understanding the model and power is a next-gen game. I don't have time for it.

Re:CLI: Improved by burntsushi on Friday August 31, 2018 @08:49AM (#17885087)
Just because I built ripgrep doesn't mean I've reinvented a wheel without understanding the existing model/power, so your criticism feels a bit disingenuous to me.

To be clear, with the current release of ripgrep, you cannot create structured objects from its output as easily as you might think. I get that it's fun to show how to do it with long shell pipelines for simple cases, but the current release of ripgrep would actually require you to parse color escape sequences in order to find all of the match boundaries in each line. This is what tools like VS Code do, for example. The --json output format rectifies that. There are other solutions that might be closer to the text format, but they're just more contortions on the line oriented output format that aren't clearly useful for human consumption, and it's much simpler to just give people what they want: JSON.

Re:CLI: Improved by needlepont on Friday August 31, 2018 @09:55AM (#17885570)
Wasn't referring specifically to you.. but to the gist of the article and the post previous to yours I believe. On the ansi escape sequences to find matches, etc...yes, I get what your tool does but having to tokenize against ansi escape codes and other ad-hoc env artifacts is something I'm glad to leave to authors that enjoy it...not that it is terribly difficult unless you decide to reinvent the wheel and optimize everything.
Re:CLI: Improved by demosito666 on Thursday August 30, 2018 @04:43PM (#17880573)
This is correct. The thing is, powershell is not for humans. It is purposed more towards configuration and system scipting, and thus should be compared to something like ansible. Using it as a shell is counter-productive unless you have very specific mindset.

Unix shell, on the other hand, is a trade-off: it offers you options to process and automate and reasonable convenience when working interactively. None of those aspects is perfect, of course, but it allows gradual learning curve (i.e. being able to do quick and dirty things from the beginning), is much more versatile and ubiquitous.

Re:CLI: Improved by krylon on Friday August 31, 2018 @10:49AM (#17886012)
At work, I always keep a PowerShell instance open. I like the fact that many common commands have Unix-like aliases, plus a lot of server software offers PowerShell interfaces for administration (e.g. Exchange, VSphere, SharePoint, ...).

On Windows, I strongly prefer PowerShell for interactive work, although I cringe a little each time I see how much memory it uses.

Re:CLI: Improved by h1d on Friday August 31, 2018 @03:55AM (#17883748)
Who is going to write "configuration and system scipting" if it isn't for humans? Ansible playbook reads very easily to humans.
Re:CLI: Improved by demosito666 on Friday August 31, 2018 @07:38AM (#17884673)
Try to run it interactively :)
Re:CLI: Improved by jniedrauer on Thursday August 30, 2018 @08:21PM (#17881975)
I disagree about objects being preferable to text in a shell. Text is very easy to reason about, and you can quickly determine what transformations you need to make based on visual feedback. Working with objects means spending a lot more time in the documentation learning what properties you have to work with.

In a programming language, it's obviously no context. But in a shell, I want to spend more time doing and less time reading.

Re:CLI: Improved by TeMPOraL on Friday August 31, 2018 @04:34AM (#17883905)
> Text is very easy to reason about, and you can quickly determine what transformations you need to make based on visual feedback. Working with objects means spending a lot more time in the documentation learning what properties you have to work with.

Or not. Take `ps`. You'll need to spend time in documentation anyway, figuring out what process properties can be shown with what flag, and then you'll be bitten later by things like the difference between `ps ux` and `ps aux` including process names in square brackets, etc. Contrast with PS equivalent, `Get-Process`. Type `Get-Process | Get-Member` to list properties of the objects returned by `Get-Process`, and you can quickly see both properties you can inspect (with descriptive names, not "VSZ" or "RSS") and what methods you can call directly (instead of extracting properties and piping to other programs).

This is IMO much cleaner, easier to work with interactively (properties instead of constant parsing and unparsing of text), better for interoperability (you're limited by what actual objects expose, not by what pieces of them a CLI program wishes to print, and if it so happens that objects somehow print more than they expose in properties, you can still call ToString() on them and get that data "the UNIX way"), and correctly separates presentation from content.

The only real drawback I've seen of Powershell is the lack of quality-of-life scripts and executables in the system. Like the md5 example elsewhere in this thread.

Re:CLI: Improved by jniedrauer on Sunday September 02, 2018 @12:47AM (#17895601)
A shell is supposed to be an interactive user interface and a very thin glue layer for some simple tasks. Once you move beyond a certain level of complexity, a shell is just the wrong tool for the job.

Powershell is too complex to be a good shell, and there are too many bizarre idiosyncrasies about it to hold its own as a programming language. It just doesn't really have a place.

Re:CLI: Improved by TeMPOraL on Sunday September 02, 2018 @06:13AM (#17896426)
Yes, the shell can be seen as a subset of OS REPL that's been optimized for efficiently performing simple system tasks and gluing things together. The problem is, ever since Lisp Machines died off and UNIX won, we've lost the REPL. We're missing a tool for doing complex tasks interactively, so people naturally started to repurpose shell for that.
Re:CLI: Improved by Nelson69 on Thursday August 30, 2018 @05:34PM (#17880997)
How should ls coloring work in your view? ls just does the file system reading and then some other tool parses the ls output matches some expressions and adds color? Or something like that?

I suppose it's just a generic colorizer tool that takes a configuration file of expressions and arbitrarily colors data feed through it?