>Generational benchmarks show big leaps in every new arch on userbenchmark,

That is because GPU scales very well with Transistor count, throw in more transistor and memory bandwidth and you are good. On the CPU you could have doubled the transistor budget and get marginal improvement in IPC.

To expand on that, this is my understanding: process shrinks used to lead to faster clocks AND more transistors, but a wall was hit and since then they only lead to more transistors. GPU's can use those extra transistors because their workload is embarrassingly parallel (thousands of parallel things), but CPU's can't because their workload is mostly sequential (developers have to explicitly write code for every thread to do something useful, and find this hard to do). You can see the clock rates peaking with the Pentium 4, crashing when that design proved a dead end, and then plateauing: https://i.stack.imgur.com/z94Of.png

IPC improvements are still happening, which is why Intel has been on 14nm for years and still makes CPU's every year that are a little bit faster, but they are hard to do and you only get incremental improvements.

If someone could figure out a way to write regular software that easily parallelized to hundreds of cores, CPU's could also get the same leaps in performance that GPU's have been enjoying. It's a software problem more than it is a hardware problem.

>If someone could figure out a way to write regular software that easily parallelized to hundreds of cores, CPU's could also get the same leaps in performance that GPU's have been enjoying. It's a software problem more than it is a hardware problem.

Amdahl's law [1] , and while we have demonstrate parallelised code can speed things up, generally speaking Software performance still have lots of low hanging fruit without even going deep into parallel and concurrency world, it is just a matter of cost whether it is worthwhile to improve or fix it.

[1] https://en.wikipedia.org/wiki/Amdahl%27s_law

Why can't a GPU power a computer?

I'm not an expert and might be horribly wrong and welcome edification, but I believe you don't want your GPU powering your computer because

- GPUs don't handle I/O other than writing to video RAM.

- GPUs don't handle interrupts.

- GPUs don't handle branching well.

For the first two, a lot of infrastructure that is part of the chipset and platform would have to be extended to each compute unit of the GPU.

Imagine a database on a GPU. It's not like 1.5K-2k+ cores that are super good at math can read and write your disk or disk array at once.

Same reason we have OLAP and OLTP systems. Same reason it makes sense for some use cases to set up a Hadoop cluster, and for others to have a single but fast CPU: some tasks can be run in parallel, others can't.

See Amdahl's law : https://en.wikipedia.org/wiki/Amdahl%27s_law

You can calculate the value of each screen pixel without knowing the value of the others. To calculate the n value of a fibonnaci sequence you have to calculate the n-1 and n-2 values first, so it doesn't matter how many cores do you have. Gpus are for the former and CPUs for the latest.

Nitpick: I understand your point in general, but it's not actually true that you need to calculate the n-1 and n-2 values first for Fibonacci.

You can use Binet's formula or matrix exponentiation to calculate it without such a dependency.

GPUs are made to execute a limited set of the same operations operation on on a huge amount of data in parallel. This is a totally different workload than your usual computer programs, that commonly have a long and complicated series of commands. Thus just translating your program 1:1 to your GPU computer would make it way slower. Your GPU runs probably around ~1.5GHz, a third of your CPUs. The GPU does have a few thousand cores that run in parallel while our consumer CPUs does have at best a dozen (that are more capable than any single GPU core), but most of those are often already idle, since it's a lot of work to make your software take advantage of them. For some tasks it's worth it, and those take advantage of e.g. you GPU.

The biggest issue is lack of huge caches, and the extreme latency to main memory due to the speed of light.

GPU’s are fine or even better for a wide range of compute tasks see CUDA etc, they just end up being very slow at a subset of common tasks.

Circuits don't operate at speed of light, though. People like to say this over and over again, but electron mobility through a circuit is only around 2/3s the speed of light, IIRC. They may have very tiny mass, but they still have mass.

No, it's the electromagnetic field which propagates at around 2/3c in the conductor. The electron drift velocity is on the order of a fraction of a mm/s.

It is the mass in the electrons that creates the heat.

Can you explain, why CPU's do not scale so well?

Because GPUs always run trivially parallelizable workloads and CPUs usually don't.

I am not an expert but I will take a shot. CPU speed is in part determined by the clock. Faster clock, more data processed per second. That's the simplest understanding. Now for modern CPUs, we use a pipeline architecture. So just like how a compiler like LLVM has a tons of optimizations, the same can be said for CPUs. Your CPU doesn't quite execute instructions one by one like a hardware Turing machine. It consumes a large chunk of instructions, try to figure out what's needed and what's not, and then process it. This is all done the few pico seconds of each cycle. It's less like a car manufacturing line and more like a fruit sorting machine, different operations gets executed simultaneously if the processor can prove that they are not e.g. required to be in order. A CPU with more transistors can easily process more optimizations i.e. bigger pipeline. You can also throw in other stuff like caches to squeeze out even more performance. The other side of Moore's law is economics, essentially cost of computing gets halved, more processing power for half the cost. Same chip, more power. Now for CPUs, clock speed has pretty much plateaued in the last decade. It is increasing, but slowly. At the same time, there are limits to pipelining, and if you get too eager with optimizations, you end up introducing vulnerabilities (since this is HN, I think I don't need to cite examples, there have been plenty). We are approaching the limit of transistor scaling and the clock isn't magically oscillating faster so there is now greater urgency on the architecture side. Current software is unable to take full advantage of multi-core processors or parallelism to the extent of making Moore's law great again. (the last time Intel tried to launch a CPU with emphasis on parallelism didn't really work out, look up Itanium). It shifts a lot of work to the compiler end and makes life unnecessarily difficult for compiler authors.

Now GPUs are designed for running multiple computations simultaneously, like CPUs, there's the fact that most operations cannot be trivially parallized. Matrices and linear algebra can because of obvious reasons — large grid of numbers, easier to process at once than with a for-loop. Most problems are not like this. The next step for computing may be FPGAs, digital circuits that can be reconfigured on-the-fly. Certain new Intel CPUs have this and most processors have a basic version in the form of microcode. Whether this will work out, I do not know. FPGAs is too more compatible with trivially parallized tasks than your usual CPU based workload. Theoretically in future you can perhaps spend 20min compiling Minecraft or Factorio to FPGA bitcode during the installation process, though more likely it is only the processor intensive bits that gets optimized. When you load the game, part of the chip gets reconfigured for it. I believe this will benefit long running processes more than short ones.

Now I am no subject domain expert so somebody with more insider experience should elaborate on this.

Number of transistors/$ is about to reach a plateau soon (if it hasn’t already.)

More powerful GPUs will happen as Moore’s Law continues for a few more years, but the free for all will soon be over.

This is true for the whole semiconductor industry, and the consequences will be very interesting, not necessarily in a good way.