News for hackers. Stuff that matters.
Re:Apple's New Mac Pro Begins Showing Up in Benchmarks by stephencanon on Thursday June 20, 2013 @06:21AM (#5911210)
Geekbench doesn’t use the GPU; Geekbench also doesn’t use AVX, so half of the floating-point performance of recent cores is left on the table.

Actually, looking at disassembly of Geekbench, they barely even use SSE vectors, so it’s really nearly 3/4 of the floating-point performance that goes unmeasured by the benchmark.

One could perhaps argue in the defense of the benchmark that many consumer applications don’t take advantage of these ISA extensions, but you definitely cannot argue that pro apps don’t use them; this alone makes Geekbench (and in fairness, most benchmarks, which tend to be pretty naive) extremely misleading for evaluation of “pro” hardware. Ultimately there’s only one benchmark that’s useful for such evaluation: actually exercising the workload that you intend to run on the machine.

Geekbench also doesn’t use available libraries that offer high-performance implementations of the computations in question (of which any serious application developer would avail themselves). Looking at numbers for released hardware, the “LU decomposition” results in Geekbench make clear that they are either using far too small of a problem size or using a naive C implementation of the operation and measuring the performance of whatever code their compiler generated. If they wanted to show the performance of the system, they would use one of the available library implementations.

Such benchmarks tell you something, but it isn’t really how good hardware is; it’s closer to a measure of how well a compiler can salvage naive C code when it isn’t allowed to take advantage of the available hardware features. I’m not sure what that’s representative of, but I know that it’s not representative of professional workloads.

Re:Apple's New Mac Pro Begins Showing Up in Benchmarks by diroussel on Thursday June 20, 2013 @07:40AM (#5911449)
I guess a LINPAK Benchmark would be more suitable? http://www.top500.org/project/linpack/
Re:Apple's New Mac Pro Begins Showing Up in Benchmarks by stephencanon on Thursday June 20, 2013 @08:08AM (#5911531)
LINPACK has its own issues as a benchmark; it only exercises extremely predictable floating-point multiply add loops. If a well-tuned implementation of the operation is used (as you see in the scientific computing community), this is representative of the peak arithmetic throughput that the hardware is capable of (so it is a good proxy for raw compute power), but it is still not representative of the way most real workloads (even professional or scientific computing) drive hardware. Even the scientific computing community is slowly starting to move away from LINPACK (see, for instance, http://www.sandia.gov/~maherou/docs/HPCG-Benchmark.pdf).

LINPACK as you typically see reported for consumer and mobile devices is far worse; the implementations used there are often naive scalar implementations that do not do any form of cache tiling. This means that for small problem sizes they are actually measuring how clever the compiler is, and for large problem sizes they are actually measuring memory throughput (both of which are interesting, but they are not what a “CPU benchmark” should be reporting).

The set of operations covered by Geekbench is actually fairly interesting (with some exceptions). If they simply used the best available libraries instead of their own naive implementations it would almost be a respectable benchmark.

Re:Apple's New Mac Pro Begins Showing Up in Benchmarks by frozenport on Thursday June 20, 2013 @03:21PM (#5914270)
Also LINPACK doesn't do IO, which a frequently sets the max speed.
Re:Apple's New Mac Pro Begins Showing Up in Benchmarks by sjtgraham on Friday June 21, 2013 @06:59PM (#5921584)
Apple also has it's own heavily optimised version of LINPACK available in Accelerate.framework