> First he doesn't know about ripgrep

ripgrep isn't POSIX compliant. Never was and never will be. So I don't think it's really applicable here. ripgrep is maybe an existence proof that a competing tool can be written, but it is certainly not a suitable POSIX compliant grep replacement. Building a fully POSIX compliant grep tool with good performance like GNU grep is pretty difficult. It could be done. It would probably take me a couple months (and only by leveraging my existing work). I just don't have any incentive to do it, personally.

> Second,none of the known grep's can find unicode strings.

That's definitely not true. GNU grep can do this just fine:

    $ echo 'Δ' | LC_ALL=en_US.UTF-8 grep '\w'
    Δ
    $ echo 'Δ' | LC_ALL=en_US.UTF-8 grep 'δ' -i
    Δ

GNU grep does pay more of a performance penalty for Unicode support than ripgrep does. It's easy to see this with a somewhat pathological case by comparing `rg '\w{42}'` with `LC_ALL=en_US.UTF-8 grep -E '\w{42}'`. (Not all cases are pathological, but ripgrep and GNU grep are both very good about literal optimizations, so it's easiest to demonstrate with a pathological case.) I'm not sure if one can attribute this to better library support in the Rust ecosystem though. It's really about how the regex machinery itself is built.

Unicode search is not searching for byte equivalents. Unicode characters can be composed in different ways, with different bytes. What you see is a á but internally it can be composed of different bytes. the single a with accent or with a mark, a plus accent.

Unicode search needs do be optimized to normalize characters, and similar problems exist for case folding, grep -i

These problems are also security relevant btw. Many unicode strings are now identifiers, like names or file paths. that the stdlib still provides no functions to search and compare strings is a much bigger beef. grep is just a symptom of a much bigger problem

Yes, you mentioned normalization and I didn't correct that, because indeed, grep can't do that. But you also just said "Unicode search," which is a pretty general term. GNU grep is certainly Unicode aware, as I demonstrated. It just doesn't solve every use case you want. But it does have Unicode-aware character classes and also Unicode aware case folding.

A grep tool that did normalization would be extremely slow. You'd probably be better served by a specialized tool. Or even better, it should be possible to write a fairly simple wrapper that puts everything into your desired normal form before searching.

- Theo doesn't give a shit on ripgrep, but grep(1) complying with standards and working under OpenBSD scripts.

- I am using grep(1) on unicode stuff in Spanish on XTerm just fine, even in ed(1). This is not a GNU craputils base, but the OpenBSD one. Heck, I type Spanish only characters with ed(1) in order to write my phlog. I can use nvi(1) from ports, but ed(1), fold(1) and ispell(1) are more than enough and I can forget about switching modes.

- That's a great example of Cargo Cult.

- On Unix and descendants, plan9 gave us UTF8, and fore sure it's grep supports more UTF8 stuff than your non-existant Rust based OS.