Both machines have a RAID 1 of the same model of SSD (2x Corsair force 3 GTs) and no PCIE involved. Each gets about 1.1 gb/s when I benchmark it in Bonnie++.

That and all 8 cores (I am counting the HT threads as cores) in the system monitor are fully saturated on both machines until the the linking stage of the build. I have a great deal of confidence the newer CPUs are significantly for at least building C++ code when disk bandwidth is not an issue.

And more available memory, letting the OS cache more files in RAM.

The older machine as 32 GB of RAM and the newer has 64gb. This could potentially be an issue, but generally the build consumes no more than 6GB leaving a leaving amount unused. Memory bandwidth is more likely.

But really I am building C++, all the parsing and re-parsing is stupidly CPU intensive. most 5400 rpm drives can feed the compiler fast enough to keep them saturated.

With a hot cache, your disk is only going to do writes (and waiting on those writes is the reason why they suggest a make -jN with N higher than the number of cores). All reads are going to be serviced from memory. All writes will also go to the cache and will be coalesced on close().

The remaining memory will be used as a crazy fast file cache. As far as 5400rpm drives that may be true if it was just a linear read, but the real problem is latency.

The rest of the RAM could be used as a file cache, but it is a moot point. 100% CPU use on each of the 4/8 cores/threads. Nothing but more CPU horsepower will significantly speed that up.

If you look at a detailed graph of CPU usage you should see a pumping action as the CPU goes full bore and then stalls out waiting for data. Filling in those stalls is what gives you the extra performance.

What is your preferred tool for gathering this kind of detailed data?

On windows It's WPR and WPA. In sure Linux/Mac has similar system wide profilers. WPA will give you detailed wait analysis.

I don't have windows. I will google for "systemwide profilers". Thank you.

I doubt this will change anything, (because the compilation time decreased as CPU time was added) but the information could be helpful in other situations.

the 2011s may not have offered it as configured by apple, but they will use 16g ram if you buy 8g dimms.

They do. Apple claims 8 GB is the max, but mine works fine with 16 GB.