In my personal tests, it's not at all orders of magnitude faster than GNU grep. In fact, for single file, non-unicode greps (most of my usage) the difference is so small as to be imperceptible when interactively using it.

Even for multi-file scenarios, the difference is nowhere near close the difference between GNU grep and BSD grep. This means that compatibility with GNU grep takes priority for me and it's not worth switching over to ripgrep.

It's a simple break down of communication. I try to be upfront about this, but no matter how hard I try or how many times I try to clarify it, I can't stop the spread of inaccurate claims. (And my goodness, this entire HN thread has numerous inaccuracies just with GNU grep alone.)

On equivalent tasks, ripgrep is not orders of magnitude faster than GNU grep, outside of pathological cases that involve Unicode support. (I can provide evidence for that if you like.)

For example, in my checkout of the Linux kernel, here's a recursive grep that searches everything:

    $ time LC_ALL=C grep -ar PM_RESUME | wc -l
    17

    real    1.176
    user    0.758
    sys     0.407
    maxmem  7 MB
    faults  0

Now compare that with ripgrep, with a command that uses the same amount of parallelism and searches the same amount of data:

    $ time rg -j1 -uuu PM_RESUME | wc -l
    17

    real    0.581
    user    0.187
    sys     0.384
    maxmem  7 MB
    faults  0

Which is 2x faster, but not "order of magnitude." Now compare it with how long ripgrep takes using the default command:

    $ time rg PM_RESUME | wc -l
    17

    real    0.125
    user    0.646
    sys     0.654
    maxmem  19 MB
    faults  0

At 10x faster, this is where you start to get to "order of magnitude" faster claims. But for someone who cares about precise claims with respect to performance, this is uninteresting because ripgrep is 1) using parallelism and 2) skipping some files due to `.gitignore` and other such rules.

You can imagine that if your directory has a lot of large binary files, or if you're searching in a directory with high latency (a network mount), then you might see even bigger differences from ripgrep without generally seeing a difference in search results because ripgrep tends to skip things you don't care about anyway.

In summary, there is an impedance mismatch when talking about performance because most people don't have a good working mental model of how these tools work internally. Many people report on their own perceived performance improvements and compare that directly to how they used to use grep. They aren't wrong in a certain light, because ultimately, the user experience is what matters. But of course, they are wrong in another light if you're interpreting it as a precise technical claim about the performance characteristics of a program.

> [...] because ripgrep tends to skip things you don't care about anyway.

so you bring the trick #1 from the grep author to a new level:

> #1 trick: GNU grep is fast because it AVOIDS LOOKING AT EVERY INPUT BYTE.

You stop looking at entire files, which can avoid a lot of bytes to go through.

Also parallelisms surely helps with a lot of files, or big ones really big ones which could be processed in chunks.

No, that's a bad trick to rely on for speed these days. GNU grep's speed comes from using memchr in its skip loop. Skipping bytes is only going to matter when the thing you're searching for is longer. For short needles, there isn't much opportunity for skipping, so it's better to say inside a vectorized routine like memchr (which, for glibc, is written in Assembly on most platforms).

Whoa, which "time" command are you using that provides memory and page fault info?

I'm using the `time` built-in from zsh, with this config in my ~/.zshrc:

    TIMEFMT=$'\nreal\t%*E\nuser\t%*U\nsys\t%*S\nmaxmem\t%M MB\nfaults\t%F'

From what I can tell, the GNU time command has a -f "%F" flag that will provide it.

http://man7.org/linux/man-pages/man1/time.1.html

Excellent tool. rg is very fast. I am using it at work now :)

Second that. Well done.

^ this. For common usage of searching for decently long patterns (>3-6 bytes) with several literal characters/limited wildcarding on a text with a largish alphabet, it is algorithmically very difficult to do _far_ better than GNU Grep’s Boyer-Moore-based implementation.

That situation changes when you have very short patterns, very long patterns, many patterns, or small alphabets (eg DNA). As burnsushi notes, ripgrep’s performance difference for common feel usage comes more from being smarter about its input than from algorithms (its algorithms are solid, of course).

So if you cripple ripgrep, to act like grep, it is not much faster. This is kind of obvious. Ripgrep is faster because it makes assumptions, that will hold true for a large part of its users.

This is like saying "if I remove wings from a plane, then it won't go much faster than my car. So a plane is not technically faster than a car"

Of course, users should know the difference between grep and ripgrep (especially with tweaks like .gitignore, which could be confusing if you don't know).

They should also know that

  git ls-files -z | xargs -0 grep -nHE <regexp>

can be almost as fast.

FYI one thing I have found that rg is much slower than GNU grep is when I have 100K exact match patterns.

Yup, that's a known bug. It will be fixed in the next release. I've already done the leg work to fix it, and it should be on master soon. My local copy of ripgrep can now search with my local /usr/share/dict/words as exact patterns just as fast as grep.

> Which is 2x faster, but not "order of magnitude."

Well, it's a base-2 order of magnitude!

There’s a reason VSCode uses ripgrep under the hood for searching. It can make a huge difference in a large (1M+ LOC) project.

This "huge diffrence" is just miliseconds for a 20M LoC, as the author showed in this thread using the linux kernel source, he gives importance to the UX when compared to grep though.

Read the second half of his comment and ask what percentage of the Linux kernel source would be excluded by file type rules versus other projects. I mostly work on web and data archival projects and ripgrep is noticeably seconds (or more!) faster because it can skip images, archives, etc.

If you’re talking about the core algorithm, it’s not that much faster but the clever adjustments for common patterns and modern CPU architectures also counts for real world results. It’s just important to be precise so people know there hasn’t been some fundamental breakthrough in searching.

For all my searches so far, ripgrep is about 30-100x faster. Perhaps you're on some hardware which ripgrep isn't well suited for.

That is simply impossible.

You are measuring the wrong thing or a case where rg can skip most files.

+1 for ripgrep. It's crazy fast

It's more that it uses concurrency intelligently. The underlying stuff is similar to GNU grep.

ripgrep's README has this to say about speed:

> ripgrep is built on top of Rust's regex engine. Rust's regex engine uses finite automata, SIMD and aggressive literal optimizations to make searching very fast. Rust's regex library also maintains performance with full Unicode support by building UTF-8 decoding directly into its deterministic finite automaton engine.

The author said in a sister thread[1] that for similar tasks the performance is comparable. ripgrep (and ag) both feel incredibly fast because they skip whole files and directories which are usually unwanted when searching.

[1]: https://news.ycombinator.com/item?id=19523263

Worth mentioning that the main author of ripgrep is also the main author of Rust's regex library, so their development nicely complements each other.