4 per socket, and at most 2 sockets per board.
Broadwell-EX, to be released this quarter, has 24 cores and up to 8 sockets per board.
So 64 FP ops per machine and cycle versus… 6144.
In the same time, GPUs went from 900 GFLOPS per card, 2 cards per machine (1800 GFLOPS total vs. 192 on CPU), to 9600 GFLOPS per card, 4 cards per machine (38400 vs. 12000). GPUs are still faster, but the advantage isn't that significant any more.
First with Haswell introducing the fused multiply add, suddenly all the 'peak flops' numbers doubled, which is technically true, but only if everything you do is a fused multiply add (with no cache misses of course).
Even so only the Xeon Phi (and only the unreleased silvermont cores?) has 16 wide vector units, even Skylake still has 8 wide AVX units, which would be 16 fma operations.
Are you saying that AVX instructions are pipelined (or some other technique) and have a throughput greater than their width per cycle?
https://www.google.com/search?q=haswell+32+single+precision+...
> First with Haswell introducing the fused multiply add, suddenly all the 'peak flops' numbers doubled, which is technically true, but only if everything you do is a fused multiply add (with no cache misses of course).
Yeah, FMA (fused multiply-adds).
Better or worse, it's de facto standard to quote one FMA as two FLOPS, because it's a very commonly combined operation.
> Are you saying that AVX instructions are pipelined (or some other technique) and have a throughput greater than their width per cycle?
Yeah, AFAIK, they're (mostly) pipelined and dual issue per clock.
Bloomfield era you could have 4 (?) cores per CPU socket. Now Broadwell EP has 22.
Only thing that hasn't scaled much CPU side is memory bandwidth. I think it's only a matter of time until Intel integrates HBM2 or something like it to same package. They've already done that for eDRAM.