I don't understand the I/O part of the whole deal. AFAIK you can't read multiple files at once off the drive, so are the files read sequentially to memory and then searched in parallel in memory? Does memory allow you to read multiple places at once, even with multiple threads? Surely the files are too large for the processor caches to be of any effect. So if you'd entertain my curiosity a bit more, is what's happening the fact that multiple files are cached in memory and memory is so fast that loading bits of the files to the processor cache takes less time than regexing those bits, moving the balance of this process to the CPU-bound side? Is the actual parallelism in the fact that multiple cores can search their individual caches at the same time and loading those caches from the RAM is fast enough to not become a bottleneck? Sorry for possibly amateurish question, I've never dug deep enough when it comes to parallelism to understand this, but I spent a good amount of time thinking about parallelising I/O stuff and came to the conclusion that I/O must be magnitudes slower and thus always is a bottleneck and any sort of I/O-bound problem (which file search surely is) must be non-parallelizable and instead can only be sped up by keeping indexes the way some OS's do.
While coding, your overall system is really just idling and it wouldn't be unusual for many of your projects header files and source files to be cached in memory because of your last compile. This effect is more pronounced on machines with a lot of memory of course. The cost savings from careful disk management is very important overall, but will it speed up the ag application? I'm surprised that ag only gets a 15% speed up with threads, but naturally it will depend on many factors.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
As an example, we have a server that can saturate 40 cores of CPU during load if fed data quickly enough. On the first load after cold boot, it takes about 2 minutes to run (fed by SSDs). Second boot, about 30 seconds.
If we run a sequential data load off SSD, it's more like 8 minutes on the cold run. So, even off non-RAM storage, parallel reads can help a lot.
Side note: originally, ag was single-threaded. When I added pthreads, it only gave a 15% speedup in my test benchmark.[1] The speedup is larger if checking a file for matches requires more CPU usage (such as when using a complex regex).
1. http://geoff.greer.fm/2012/09/07/the-silver-searcher-adding-...