A lot of comment mentioned about IPC, but something obvious no one has mentioned ( yet ).

For Server is it also about core count. For the same TDP AMD offer 64 Core option. On a Dual Socket System that is 128 Core. Zen 3 Milan is also socket compatible with Zen 2 Rome.

Basically for Server Intel has been stuck on 14nm for far too long. The first 14nm Broadwell Xeon was released in 2015, and as of mid 2021 Intel barely started rolling out 10nm Xeon Part based on IceLake.

That is half a decade of stagnation. But we are now finally getting 7nm server part, 5nm with Zen 4 and and SRAM Die Stacking. I am only hoping EUV + DDR5 will bring Server ECC DRAM price down as well. In a few years time we will have affordable ( relatively speaking ) Dual Socket 256 Core Server with Terabyte of RAM. Get a few of those and be done with scaling for 90% of us. ( I mean the whole StackOverflow is Served with 9 Web Server [1] on some not very powerful hardware [2] )

[1] https://stackexchange.com/performance

[2] https://nickcraver.com/blog/2016/03/29/stack-overflow-the-ha...

instructions per clock have been going up, l3 cache sizes going up, core count going up, new wider instructions becoming available (avx 2 and 512 ), branch predictors improving.

overall speed increases are still nothing like the old days and spectre style mitigations have been eating away at the improvements

Yes. Without a L1 cache a cpu is just a hyper active kid in a box. L2 important too. If one cares about performance learn about caches.

Clocks aren't increasing much, but IPC still is. Density is also still increasing meaning lower latencies, lower power and therefore higher performance per area.

IPC is going up but slowly compared to previous generations. If AMD can sustain the kind of generation on generation increases it achieved between Zen 2 and 3 I will be tremendously impressed, but as it stands my brand new 5600X has barely 50% more single-core performance than the seven year old 4670K I replaced.

I have a hunch that it would be much greater than 50% if the software was recompiled to take advantage of the newer instruction sets. Then again, you won't be comparing exact binary copies of software, which some people might say invalidates it as a benchmark/comparison.

I remember many, many years ago my PC couldn't run the latest adobe software as I was missing SSE2 instructions on my then current CPU. I believe there was some hidden compatability mode, but the software ran extremely poorly vs. the version of software that was out before that didn't require SSE2.

There have been almost no new broad "performance-oriented" instruction sets introduced since the Haswell era, all the way to the Zen 3 era (e.g. SIMD/vector) for compilers to target. At least not any instructions that the vast majority of software is going to see magical huge improvements from and are wide spread enough to justify it (though specific software may benefit greatly from some specific tuning; pext/pdep and AVX-512 for instance.)

The microarchitectures have improved significantly though, which does matter. For instance, Haswell-era AVX2 implementations were significantly poorer than the modern ones in, say, Tiger Lake or Zen 3. The newer ones have completely different power usage and per-core performance characteristics for AVX code; even if you could run AVX2 on older processors, it might not have actually been a good idea if the cumulative slowdowns they cause impacts the whole system (because the chips had to downclock the whole system so they wouldn't brownout). So it's not just a matter of instruction sets, but their individual performance characteristics.

And also, it is not just CPUs that have improved. If anything, the biggest improvements have been in storage devices across the stack, which now have significantly better performance and parallelism, and the bandwidth has improved too (many more PCIe lanes). I can read gigabytes a second from a single NVMe drive; millions of IOPS a second, which is vastly better than you could 7 years ago on a consumer-level budget. Modern machines do not just crunch scalar code in isolation, and neither did older ones; we could just arbitrage CPU cycles more often than we can now in an era where a lot of the performance "cliffs" have been dealt with. Isolating things to just look at how fast the CPU can retire instructions is a good metric for CPU designers, but it's a very incomplete view when viewing the system as a whole as an application developer.

I'm not deep on the weeds of high-performance compilers, but just because ISA evolution hasn't happened doesn't mean compilers can't evolve to use the silicon better.

There always was an "Intel advantage" to compilers for decades (admittedly Intel invested in compilers more than AMD, but they also were sneaky about trying to nerf AMD in compilers), but with AMD being such a clear leader for so many years, I would hope at least GCC has started supporting AMD flavors of compilation better.

Anyone know if this has happened with GCC and AMD silicon? Or at least is there a better body of knowledge of what GCC flags help AMD more?

Yes, I consider this to fall under the umbrella of general microarchitectural improvements I mentioned. GCC and LLVM are regularly updated with microarchitectural scheduling models to better emit code that matches the underlying architecture, and have featured these for at least 5-7 years; there can be a big difference between say, Skylake and Zen 2, for instance, so targeting things appropriately is a good idea. You can use the `-march` flag for your compiler to target specific architectures, for instance -march=tigerlake or -march=znver3

But in general I think it's a bit of a red herring for the thrust of my original post; first off you always have to target the benchmark to test a hypothesis, you don't run them in isolation for zero reason. My hypothesis when I ran my own for instance was "General execution of bog standard scalar code is only up by about 50-60%" and using the exact same binary instructions was the baseline criteria for that; it was not "Does targeting a specific microarchitecture scheduling model yield specific gains." If you want to test the second one, you need to run another benchmark.

There are too many factors for any particular machine for any such post to be comprehensive, as I'm sure you're aware. I'm just speaking in loose generalities.

> You can use the `-march` flag for your compiler to target specific architectures, for instance -march=tigerlake or -march=znver3

Note that -march will use instructions that might be unavailable on other CPUs of the target. -mtune (which is implied by -march) is the flag that sets the cost tables used by instruction selection, cache line sizes, etc.

I recently watched the CppCon 2019 talk by Matt Godbolt "Compiler Explorer: Behind The Scenes"[1], and a cool feature he presented is the integrated LLVM Machine Code Analyser tool. If you look at the "timeline" and "resource" views of how a Zen 3 executes a typical assembly snippet, it is absolutely mind blowing. That beast of a CPU has so many resources it's just crazy.

[1] https://www.youtube.com/watch?v=kIoZDUd5DKw

march=native

I doubt that very much. I have a i7 4790 and I have Ryzen 7 3700X. In my testing single core speed is nearly 2.5 times in favor of my Ryzen. What where you using as bench marks?

My test was a single threaded software rasterizer with real-time vertex lighting. That I compiled in vs c++ 2008 around 2010.

Basic single-core scalar-only workloads of my own corroborate the grandparent, as well as most of the other benchmarks I've seen. My own 5600X is "only" about 50-60% better than my old Haswell i5-4950 (from Q2 '14) on this note.

But the scalar speed isn't everything, because you're often not bounded solely by retirement in isolation (the system, in aggregate, is an open one not a closed one.) Fatter caches and extraordinarily improved storage devices with lots of parallelism (even on a single core you can fire off a ton of asynchronous I/O at the device) make a huge difference here even for single-core workloads, because you can actually keep the core fed well enough to do work. So the cumulative improvement is even better in practice.

Now I’m curious, is this testing against some requirements for an application? Are they any applications that would benefit solely from scalar performance vs scalar, cache size, and memory speed.

Need to point out in particular that that best Haswell i5 was FASTER than the best i7 for single-core workloads, so this might be a factor in OP post.

And also that is the reason why I use such processor on my own computer, it was already "outdated" when I bought it, but since one of the things I like to do is play some simulation style games that rely heavily on single-core performance, I choose the fastest single-core CPU I could find without bankrupting myself. (the i5 4690k that with some squeezing can even be pushed past 4ghz, it is a beastly CPU this one)

Exactly. The only thing I used my system for that was really begging for more CPU was console emulation, and that depends more on single core performance than anything else.

Haha I’m in almost the exact same boat, I replaced a haswell i5 with a 5800x. I definitely went overkill with my system and still haven’t gotten a GPU upgrade yet due to cost/laziness.

Indeed. I feel like I could've gotten most of the speedup by wiping windows and reinstalling but I had to make a clean break from Windows 7 or I was never going to stop using it. Hopefully GPU prices come down soon.

Even crazier, my i5 750 (from 2009) was overclocked to 3.5ghz and per core Ryzen 3 isn't even twice as fast.

We're near the end of IPC scaling per core. And it was never that good in the first place. Pentium 3 IPC is only 3-4x worse than the fastest Ryzen. Most of our speed increases came from frequency.

IMO we need to get off silicon substrate so we can frequency scale again.

I wonder if the end of scaling will push everyone into faster languages like Rust. You can't sit around 2 years for your code performance to double anymore. Will this eventually kill slow languages? I think so.

Hardware increases of speed began to slow down 20 years ago and people are still using software that is 50x slower than the fastest possible technology. If this alone was going to kill slow languages one would suppose it would have already done so.

The arguable shift that is more significant is not about hardware its hopefully about newer languages like Rust making this performance cost less in terms of safety and development time which is a more recent development.

I agree partially. But excuses for using dog slow languages, at least where I've worked, revolved around "it will be twice as fast in two years anyways"

It's clear that's no longer true, which gives less excuses for using say Ruby over Java.

Clockrate is not a bottleneck on how fast your computer is. It's just a synchronization primitive.

Think about it like the tempo of a song. The entire orchestra needs to play in sync with the tempo, but how many notes you play relative to the tempo is still up to each player. You can play multiple notes per "tempo tick".

You will have to use a faster internal clock to play those faster notes, though.

The point is that you don't. A clock tick is not the smallest unit of operation. It's the smallest unit of synchronization. A lot of work can be done in-between synchronization points.

IPCs. Instructions per clock.

We haven't had a strict clock speed = performance ratio for over a decade, it's just one component of it now.

https://en.wikipedia.org/wiki/Cycles_per_instruction

https://en.wikipedia.org/wiki/Instructions_per_second

Rather than just bumping clock speed, they've made improvements on what can be done within the clock cycle.

I recently did a desktop rebuild.

Went from a Ryzen 7 2700X to a Ryzen 5 5600X.

On paper, this looks like a downgrade. After all, the 7 is higher than a 5, right? I have two more cores with a 2700X...

However, the 5600x has about a 28% IPC gain over the 2700x in single core performance despite running at the same base clock speed. Literally 20%+ faster in the same tasks even when taking the turbo boost out of the equation.

The 2700x was at 12/14nm process while the 5600x is at 7nm which also helps with the power consumption as the 5600x has a 40 watt lower TDP.

Since what I need is better single core performance over more cores, the 5600X is quite an upgrade despite only being two years newer. (A LOT has happened in 2 years with AMD) With two less cores but significantly higher single core performance, the 5600X outperforms the 2700X on both single core and multicore performance.

Unfortunately for Intel, they did a lot of little tweaks and cheats to gain performance at the expense of security and now the mitigation patches pulls their chips back to the Bulldozer era in terms of current performance. They were also stuck at the same node for almost a decade now (Broadwell, 2014), so they had no gains from a shrink (speed of light propagation gain from a shrink, less heat and higher clocks also).

My i7-4790k is full of jank and microstuttering now, it has become unusable. Imagine how the servers might be doing if (and they should be) they are being kept up to date on security and microcode patches.

Then there also comes with spec upgrades within the processor generations. Ryzen 2700X to 5600X also means PCI Express went from 3.0 to 4.0... Not hugely significant among desktop users, but substantial for servers that need that amount of link bandwidth for compute cards and storage.

TLDR: Magic.

i think for the data center the key number is performance per watt. With smaller the power requirements go down. As the power goes down, heat goes down, so you can push the processors harder.

There are of course architectural improvements that improve performance with space to add gates etc, but when you have 1000s of chips cost of energy is the big thing.

Less heat = more cores