`grl` is really just a demo program for the 8-bit clean message passing variant in `procpool` (similar to Python's multiprocessing module). `procpool` uses parent-kid pipes for its communication. The pselect6's come from using select on those N pipes. And, yeah, a lot of selects are expected since the filename passing-answer receiving happens a lot with a lot of files. { Yes, yes...I know past 1024 fd's select will be a problem, but TSMC only just hit that 1nm mark. So, we probably still have a few years before I personally can access 1024 core-thread machines.. ;-) ;-) }
mmaps and threads was discussed already in [3] linked in my first post of this thread with a brief summary. To recap briefly - since threads share all memory and since mmap alters those page tables, it is plausible the kernel just locks the whole process out of simplicity, blocking execution of other threads. It is also possible OS-? can use devious tricks/semantics to avoid that locking, but I am unaware of an exhaustive survey or proof of impossibility. Because in `procpool`/`grl` the kids which are what do the per-file mmaps, are their own processes there is no suspension of the other kids when they mmap. So, they can realize the faster IO (from SIMD register use, done by Linux for context switch optimization or "performance in the large/system-wide"). This is (perhaps) why your last test has better performance without any parallelism. I had meant to test this last time we discussed mmap & threads, but didn't at the time. `grl` is just a very preliminary test along those lines.
This is all at least partly theoretical. Someone ought to study it and write a nice blog article about it. I do not have that kind of time right now (or a blog). Or, for all I know, nice academic paper(s) already study this somewhere. I have not looked. Sadly, as old as this thread is, the general HN pool will probably not see any of this to crowd source such wisdom with direct pointers for you. I almost missed your question in my old threads checking, and I have kind of used up my time budget for this right now. You can email if you want. I get better notifies that way.
Oh, I think I missed the fact that grl is spawning processes and not threads. That's very interesting.
With respect to extra syscalls, ripgrep is actually doing a lot more stat'ing than you might expect because it is "recursive" by default. So for each file it gets, it has to at least check if it's a directory. And there's some symlink behavior to handle. I think I can get rid of one of them per file given, and others are harder because of abstraction boundaries. Sigh. Anyway, that could be impacting an I/O-only comparison here as well.
And I used
since including the C files caused the argument list to be too long on my system.My CPU is a bit dated, but is a i7-6900K @ 3.2 GHz.
My Nim version:
With those caveats, this is what I get on my system: We can somewhat control for substring search by using 'zzzzzzzz'. It won't control for everything, but it should be close I think. If I run with jobs=1 for both tools, I get: Here's where it gets interesting. If I force ripgrep to use memory maps, then I get (first with parallelism, and then without): I think on the one hand, I'm confused as to why I can't reproduce your result. But the more interesting thing to me is why ripgrep is so slow when it uses memory maps, but your program is not.The strace output for ripgrep also shows more syscalls than I would expect, so I'll be investigating that as well.
There are also lots of 'pselect6' syscalls in your program. Do you know what those are from?
Anyway, thanks for the interesting benchmark! Some interesting bits to investigate!