> it inter-operates with other tooling in a Unix-y way

Other tooling supports user locales, ripgrep doesn't. So no, it doesn't.

A tool doesn't need to support locales to be used in a shell pipeline. The Unix philosophy and shell pipelines were born long before POSIX locales. I even specifically contrasted this with POSIX compatibility. So it seems to me like you're disingenuously interpreting my words at this point.

@dang more trolling

Of course it does. If the tools it interacts with do, it has to. It's like a game of telephone and ripgrep is the kid who can't say r's

That isn't true. ripgrep will happily pass through data it gets as-is. It just itself doesn't support locales. Commands do not need to have locale support to be used in shell pipelines. `rg foo | rg -v bar` is an example that has nothing to do with locales but demonstrates that ripgrep can be used in shell pipelines.

It is. Consider

    foo | grep -E '^.{4}$' | bar

If foo prints $'O\u011Fuz' and LC_ALL is set to a valid UTF-8 locale, bar will receive what foo printed. If foo prints $'O\xF0uz' and LC_ALL is assigned a valid ISO-8859 locale, again, bar will receive what foo printed; because foo, bar, and grep are compatible. You can't have that with ripgrep; you'll have to modify the shell script in between or worse, manually parse LC_ALL and hope ripgrep supports the encoding

I found this interesting, so I tried to test it:

On LC_ALL=en_US.UTF-8:

    $ echo $'O\u011Fuz' | grep -E '^.{4}$'
    Oğuz
    $ echo $'O\u011Fuz' | rg '^.{4}$'
    Oğuz

On LC_ALL=en_US.ISO-8859-1:

    $ echo $'O\xF0uz' | grep -E '^.{4}$'
    O�uz
    $ echo $'O\xF0uz' | rg '^.{4}$'

It strangely doesn't find anything at all:

    $ echo $'O\xF0uz' | rg '^.*$' | wc -c
    0

It only does once the $ anchor is removed:

    $ echo $'O\xF0uz' | rg '^.*' | wc -c
    5

It's not strange because ripgrep doesn't understand non-UTF-8 data (unless there's a UTF-16 BOM, in which case, ripgrep will automatically understand it). But you can tell it to:

    $ echo $'O\xF0uz' | rg -E iso-8859-1 '^.{4}$'
    Oðuz

The person you're responding to has been trolling in this thread (and others) by twisting words and claiming multiple false things. When I've fixed their errors, they don't acknowledge them as mistakes and just keep on twisting words.

First of all, that doesn't invalidate literally anything I said. I have never claimed that ripgrep supported locales. I only said that it could be used in shell pipelines. That doesn't mean it can be used in all shell pipelines in exactly the same way that grep can. I even clarified this explicitly. Which is why you are a troll.

Second of all, you're just wrong. ripgrep uses UTF-8 by default, but you don't have to use it. And when you switch your locale to ISO-8859 (who uses that!?!? why does it mater!?!?), you are no longer emitting UTF-8. As you obviously know or else you wouldn't have been able to come up with the example.

As has been pointed out elsewhere in this thread, you can disable Unicode mode in ripgrep:

    $ echo $'O\xF0uz' | LC_ALL=en_US.ISO-8859-1 grep -E '^.{4}$'
    Ouz
    $ echo $'O\xF0uz' | rg '^(?-u:.){4}$'
    Ouz

Or just specify the encoding you want:

    $ echo $'O\xF0uz' | rg -E iso-8859-1 '^.{4}$'
    Oðuz

ripgrep doesn't do this the same way that grep does. But you can achieve it, which is perfectly in line with what I said.

> you can disable Unicode mode in ripgrep

And everything becomes a byte, that's not always what you want

    $ export LC_ALL=tr_TR.ISO-8859-9
    $ echo $'O\xD0UZ' | grep -E '^[[:upper:]]{4}$'
    O�UZ

> Or just specify the encoding you want

And what if I want to support more than one encoding? Am I supposed to modify my shell script every time I run it?

I'm sure you can get creative. :-) You can set an environment variable to control the encoding, expose a flag or any one of a number of other things to control the encoding. Either way, now you're just shifting the goalposts.

You've also continued to ignore my most substantive rebuttal: that a specific example where ripgrep is not compatible with grep or doesn't behave the same doesn't mean it can't be used in shell pipelines. I use ripgrep in shell pipelines all of the time. As do many others. Literally nothing you've said has invalidated anything I've said. All you're doing is finding things that some implementations of grep can do that ripgrep (intentionally) cannot do in exactly the same way. But that's fine, because ripgrep was never, isn't and will never be compatible with grep: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos...

So if you need grep compatibility get a fucking clue and just use grep.

> You can set an environment variable to control the encoding

Yes, that's what LC_ALL is. Every other tool understands it except ripgrep. Even if you parse it by hand, there's no guarantee that ripgrep will support the encoding.

ripgrep intentionally doesn't understand it. It is far from the only tool that doesn't. For example, busybox doesn't either:

    $ echo $'O\xF0uz' | LC_ALL=en_US.ISO-8859-1 busybox grep -E '^.{4}$'

So that's yet another lie from you.

If you want or need LC_ALL support, then don't use ripgrep. It's right there in the FAQ entry as a reason to use grep instead:

> Do you care about POSIX compatibility? If so, then you can't use ripgrep because it never was, isn't and never will be POSIX compatible.

Maybe learn how to read. Your complaint boils down to "ripgrep doesn't do this thing that it says it doesn't do." What a fucking revelation.

>busybox

Apples vs. oranges. Busybox is a suite of minimal tools for low-resource systems, lack of QoL features you wouldn't need in such an environment is its selling point. Besides you won't see anyone claiming individual busybox tools can be used in shell scripts just fine.

It's easier to admit ripgrep is rather a supplementary tool that is incompatible with the tooling your typical Unix-like operating system provides out of the box than a "modern replacement" for grep and move on.

So you lied. Now you've shifted the goalposts after being called out on it. And ripgrep has no problems being used in shell pipelines. What ripgrep has a problem doing is being drop-in compatible for grep, because that isn't its goal.

> "modern replacement" for grep

I have literally never called ripgrep this. So more lies and straw-manning from you. ripgrep's repo neither positions itself as a replacement or as "modern." And in the FAQ about "replacing" grep, it makes a nuanced claim that accounts for all of this. It specifically says you should use grep when you need the features of grep that aren't in ripgrep.

> OK, so you phrased your criticism very narrowly toward "lots of these Rust CLI tools," but in actuality, your criticism applies way more broadly than that.

Obviously. The topic was Rust CLI tools, that's why my post focused on them. Indeed, my criticism is broader.

> Likely to just about all of GNU's tooling, for example. And probably BSD's and busybox's tooling too. Even busybox isn't above implementing POSIX+some, and AFAIK none of them provide a strict "POSIX only" mode.

This isn't about POSIX or not POSIX, this is about introducing yet another pure convenience feature for something that can easily be achieved with other accompanying (guaranteed to be on the concerned platform) tools. POSIX doesn't create features anyway, it's about describing things that have already become standard; the Austin Group bug tracker is quite explicit about this.

> No... it doesn't? ripgrep being compatible with the Unix philosophy is not the same as it being POSIX compatible.

Well, just a question of interpretation of "Unix-like goodness". UNIX is both a philosophy and a portability standard (since POSIX == SUSv4). And as I said, doing one thing and doing it well is another facet of that philosophy; finding files != searching in them.

> And besides, I don't know of any grep that has such a mode. How do you know you aren't using a grep flag that isn't available in other grep implementations?

Compatibility != strict rejection of anything else, quite obviously.

> Nope, I don't. I've been a vim (now neovim) user for decades. And neovim even uses ripgrep by default over grep if it's installed.

Which is why I said vim instead of neovim (even if neovim is clearly better).

------

I think you misunderstand my position because I like a bit of snark: compromising on the "do one thing" part of the UNIX philosophy isn't necessarily something horrible, but it's not an imaginary issue either. I could give other examples (like how the pretty clean xsv became https://github.com/dathere/qsv) but it really doesn't matter because I think it's more of a popular CLI "apps" thing than Rust alone, even if there's community overlap.

I can no longer see any coherent point you're trying to make.

My point is that you are treating an ideal ("do one thing and doing it well") as an end instead of a means. Moreover, you are treating it as if there is any actual choice in the matter. In reality, the tooling most of us use on the command line violates the Unix philosophy in some way or another. So the question is not "should I adhere to the Unix philosophy?" but "what makes a good user experience?"

I don't see why my point is that hard to see: the "do one thing" part of the UNIX philosophy is a good ideal to aim for, even if often ignored in favour of a more convenient kitchen sink approach. I then just mentioned that a good majority of these useful Rust tools (I really like hyperfine and oxipng, for example) don't seem to care much about it.

Some example of that idea I use every day is https://codemadness.org/sfeed-simple-feed-parser.html, where feed parsing, serialization and visualization are completely separated in small orthogonal binaries with very few options. It could have been one fat "sfeed" or something like its newsboat instead. (shilling it a bit more there: https://world-playground-deceit.net/blog/2024/10/sfeed-setup...)

I’ve also been guilty of insisting on this, even when, as you point out, most tooling doesn’t strictly follow it anyway. I think it’s an overreaction from being irritated at seeing app after app fall to “now including AI,” or just pivoting for new features because it wasn’t making money fast enough.

FWIW, I’ve been using *nix for about 25 years, and had zero problems adapting to ripgrep. You kept -i and -v, which honestly covers 90% of common usage.

Sorry for the late reply. I just wanted to say that I don't think AI has anything to do with this. I've been seeing this sort of complaining, goal post shifting and confusing means with ends for probably decades now. I mean, AI was around then too, but not like it is today, which is what I assume you meant. Even when I release ripgrep ~9 years ago, AI wasn't what it was today, and I saw the same sort of complaints.

I don't usually like doing this, but if I had to step out of my wheelhouse and guesstimate, I think it's probably just some form of nostalgia reasoning. People remember the "simpler" times and just want to have it again. But of course, the "simpler" times are less a reflection of reality and more a reflection of your perception of the world. IMO anyway. (I have the same sort of inclination, but I try to recognize it for what it is: personal bias. Others seem unable to do this and mistake this personal bias for objective reality.)