News for hackers. Stuff that matters.
Re:Why GNU grep is fast (2010) by burntsushi on Monday July 25, 2022 @08:33AM (#32223443)
Author of ripgrep here.

There are two ways to talk about "speed." On the one hand, we have the "speed" that is associated with the user experience. That is, given some problem the user cares about, how fast can you solve it? On the other hand, we have the "speed" that is associated with doing precisely the same task and measuring which tool does it faster.

ripgrep is generally faster at both of those things, but it's the former where it really shines:

    $ git remote -v
    origin  [email protected]:nwjs/chromium.src (fetch)
    origin  [email protected]:nwjs/chromium.src (push)

    $ git rev-parse HEAD
    5d32cab40f738932eddc017980e2e409c5abef2c

    $ time rg 'Xvfb and Openbox' | wc -l
    1

    real    0.289
    user    1.526
    sys     1.731
    maxmem  87 MB
    faults  0

    $ time grep -r 'Xvfb and Openbox' ./ | wc -l
    1

    real    5.405
    user    3.489
    sys     1.890
    maxmem  11 MB
    faults  0
(I ran these commands multiple times each until the times stabilized. i.e., The directory tree is in cache.)

We're talking about an order of magnitude improvement here to get the same results. And not just in a "ripgrep took 10ms and grep took 100ms, but both are fast enough" sense. This is the difference between "near instant results" and "this is taking annoyingly long."

Now of course, from the perspective of the second idea of speed, this isn't an apple-to-apples comparison. GNU grep is actually searching a lot more data here. We can make ripgrep search the same amount of data as GNU grep quite easily:

    $ time rg -uuu 'Xvfb and Openbox' | wc -l
    1

    real    2.538
    user    2.570
    sys     3.017
    maxmem  72 MB
    faults  0
So, still a big improvement over GNU grep, but it's not quite as jaw dropping.

The important bit here is that a lot of people care about the improvement at the UX level. You can, for example, get a fair bit of improvement with GNU grep with some extra flags:

    $ time grep -r --exclude-dir='.git' 'Xvfb and Openbox' ./ | wc -l
    1

    real    1.630
    user    0.781
    sys     0.826
    maxmem  11 MB
    faults  0
And now you've got to shove that stuff into an alias or a wrapper script. Which... is fine. I did it for a very long time before I wrote ripgrep. I had a whole bunch of aliases and wrapper scripts, many of which were specific to certain types of projects. But once I built ripgrep, all of those aliases and wrapper scripts went away. Because ripgrep's heuristics for smart filtering by default subsumed all of them.

Finally, it's worth pointing out that ripgrep isn't intended to replace grep. It literally can't. It's not POSIX compatible. So if you're writing shell scripts and care more about portability, then 'grep' is a fine choice. Indeed, I still use 'grep' for precisely that purpose. See: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#pos...

Re:Why GNU grep is fast (2010) by heywoodlh on Monday July 25, 2022 @09:22AM (#32223879)
This is an awesome explanation! Along with other comments I feel like I now understand where I would (and would not) want to use ripgrep.

And thanks for making an awesome, open source tool. :)