> - Why do compilers not strive to simplify their code-gen phase, or enable themselves to do advanced instruction-level program analysis, or both?
Because specialized instructions formalize invariants and constraints on behavior that allow efficient computation, often by specialized hardware.
> - Why do microprocessors not strive for simplicity, implement only a handful of instructions in an optimized way, with a very small chip footprint, to be followed by proliferation of cores (think 256-core, 512-core, 1024-core).
Some do, see GPUs and coprocessors like the Phi. We don't take this approach with CPUs because real problems often require complex, branching, inhomogeneous computation, which require the type of specialization and tradeoffs mentioned above.
You're making broad generalizations, ironically speaking. If there is anything the article of this thread suggests, it is that we have created a needlessly complex instruction set, and that "efficient specialization" is not valuable to the software 99.99% of the time.
> Good design is not about simplicity; it's about managed complexity.
Managed complexity is simplicity. And it's pretty clear today's compilers and today's microprocessor designs are anything but good managers of their complexity.
The only thing the article suggests is that the author's /usr/bin only includes about 2/3rds of possible x86 instructions, and that a few instructions occur most. The latter is unsurprising and expected in almost any grammar or formal system. The former is also expected given that most specialized instructions are for specialized use in high-performance applications, which are not likely all to be in any individual workstation's /usr/bin.
The cruft of specialization is legacy instructions, which as many other comments in this thread point out are usually implemented through microcode only. They don't contribute to complexity of the processor (which can choose to inefficiently implement this with other microinstructions) or the compiler (which can choose to not emit them).
I'm all for new blood in general-purpose architectures, but the bloat and inelegance of x86 isn't really something that's mattered for at least a decade.
Sounds like the English language.
The words used in this discussion, btw – like instruction set – are also rarely ever used in English, but still necessary.
Because that would not make CPUs faster or cheaper.
If you think that modern CPUs are large because they implement a lot of instructions, you are completely wrong. The entire machinery needed for executing more than a couple of different instructions is less than 1% of the core. The space is not taken up by decoding tables or ALUs, it's taken up by forwarding networks, registers, and most of all, cache. And all of those are things very much required to make a CPU fast. CPUs have a lot of instructions precisely because the cost of implementing a new instruction is negligible compared to the size of the CPU.
Modern CPU designers have such a larger transistor budget than they need to get creative to make use of all of it, so specialized instructions are pretty much free.
And no, you can't just stuff 1024 cores in a processor; apart from the fact that most software wouldn't know what to make of it, such a monster might end up bottlenecked by intercommunication or memory bandwidth.
Also simple CPUs are just slow; you need a lot of machinery to perform well at high frequency/high memory latency.
... because we (the chip designer) are okay with larger footprint per core.
> specialized instructions are pretty much free
... only after we have fixed the footprint per core. But if we're willing to vary that parameter, then the specialized instructions are not free.
Not to mention, the main article of this thread is a strong evidence that those specialized instructions are almost never used!
As for your point about 1024 cores, the whole point I'm trying to make is that whatever software does today with 4 cores in a 4-core processor, could be done by 4-cores in a 1024-core processor, because those 4-cores don't implement the instructions that are not needed. And that means you have 1020 cores free sitting in your microprocessor. You could only make your computations faster or at the same speed (in the worst case) in their presence, not slower.
> simple CPUs are just slow
I would like to see any source of this claim. The only reason I can think of is that complex CPUs implement some instructions that help speed up. But as we can see in the original article of this thread, software is not making use of those instructions. So I don't see how a simple CPU (that picks the best 5-7 instructions that give turing completeness, as well as best performance) is any slower.
Note, by a simple CPU, I'm not advocating eliminating pipelines and caches, etc. All I'm saying is that once you optimize a CPU design and eliminate redundancy as well as the requirement of backward compatibility, you can get a much better performing CPU that what we have currently.
Here's an example of die area: http://farm6.staticflickr.com/5321/9104546631_4c7a4a023b_o.j... I don't think it's the best picture, but it's surprisingly hard to find these for recent x86 processors.
If you want to stick more cores on a single die, you have to shrink the size of a core. And looking at die space, the real big wins are the caches, out-of-order execution, and branch predictors--losing all of which will kill your IPC. The other problem with high core counts is memory bandwidth. The instruction fetch bandwidth alone on 1024 cores runs to GB/s. Cache coherency traffic and core communication traffic could likely suck up the rest of the bandwidth.
Proposing to rip out most of the instructions leaves you with the problem that you're ripping out hardware-accelerated instructions, and you have nothing to compensate for the lost speed. You can't increase clock frequencies (power will kill you); you can't increase core counts (the memory bandwidth is insufficient, not to mention Amdahl's law).
In a recent, high frequency, aggressively OoO CPU, a large part of the die is used by caches, the register file, vector ALUs and the OoO machinery; by comparison scalar ALUs and especially decoders do not take a lot of machinery. In particular microcode for all legacy instructions takes only a tiny amount of space. Legacy support might have a cost on tiny embedded processors, but it just doesn't matter in a large desktop/server class CPU.
And yes, some of those specialized instructions are rarely used (there is a reason they are called dark silicon), but it means that Intel (or ARM, IBM) doesn't need to spin a new CPU for a specialized workload. Intel is even rumored to add custom instructions required by large customers (FB, Google) on all its CPUs, which are left disabled for other customers.
A good part of the complexity of current cpus comes from features like branch prediction, speculative execution and so on so removing features wouldn't make them drastically simpler. Many of the truly rarely used instrucctions aren't build in hardware and therefore don't contribute to the complexity of the cpu anyway. Others are rarely used but add huge speed boosts for important special purpose tasks, think os kernel.
The article doesn't show that or even come close to it. To measure what instructions software uses, you'd need to measure it at runtime.
The metric of simply counting frequency on disk ignores loops: you might have a single server binary that contains a single AES-NI instruction and conclude this instruction was useless and never used, but if your server spends 50% of its time decrypting SSL connections at high speed, then things look rather different.
> - Why do microprocessors not strive for simplicity, implement only a handful of instructions in an optimized way, with a very small chip footprint, to be followed by proliferation of cores (think 256-core, 512-core, 1024-core).
What you're describing is a GPU--revert a core to a basic ALU and make it be 2 dozen 16-wide SIMD ALUs. Not everything can work well on a GPU, though; you need very high parallelizibility, and fairly uniform memory access models. It's why supercomputers switched from SIMD to MIMD models a few decades ago. (They're starting to revert to include more GPU-like cores, mostly for power reasons).
I would say I'm describing something halfway between a CPU and a GPU. It's not just an ALU, it's a complete microprocessor, with pipelining, caches, etc. The main difference is that the instruction set is optimized, backward compatibility is no longer a requirement, and redundancy of the architecture is eliminated.
It's fun to play with. The challenges are pretty much what people have been saying: To get to these core counts in the little space they have had to sacrifice cache size and memory size and single thread speed, even with a very clean and simple instruction set, and we are in general not good about taking advantage of high core counts other than in the GPU sense.
The Epiphany in variations with enough cores has the potential to beat GPUs for workloads with many independent instruction streams, but it'd gets crushed by GPUs of similar size for workloads that can be easily vectorised, and would crushed by any modern CPU for workloads that require high single core performance because of data dependencies, no matter the core count, exactly because most of the complexity it sacrifices cost single core performance.
The problem is that we don't really know how much space that leaves (the Parallella was designed largely as a development platform to let people experiment with an Epiphany CPU). Adapteva has focused on the low power usage of their CPU, and that may very well be a good idea.
Though it is an awesome architecture and I recommend anyone to check it out.
GPUs have only something like 1-32 independent very wide MIMD/SIMD units with a lot of hardware threads.
Not sure about Nvidia 1080, but at least 980 has 32 SMM units, each roughly as defined above.
1024 independent cores... that'd be crazy. Surely they would not be cache coherent but only able to talk to immediate neighbors.
It'd be understatement to say that programming that kind of CPU would be challenging...
I didn't go for the absolute minimum, but I did aim for useful and reasonable minimum with the ggx[1] ISA (now called moxie[1]) by defining the ISA incrementally and only adding instructions used by GCC.
The approach I took is described here: [1] https://github.com/atgreen/ggx
I think it's safe to say that about 5-7 instructions are all that's needed to perform all computation tasks.
One instruction is needed to be Turing-complete. It's not very practical though, as you need many more simple instructions to do the work of a single complex one.
Consider memcpy(), one of my favourite examples. On a simple architecture like MIPS, the software has to explicitly perform each read and write along with updating the pointers, the address calculations, and the loop control. It also has to take into account alignment and when to use byte/word/dword accesses. This all requires instructions, which occupy precious cache space (and unrolling makes them take even more) and have to be fetched, decoded, and executed. It can only read and write e.g. 32 bits at a time, because that's the only size the architecture supports for individual load and store instructions. If a wider (e.g. 64 bit) revision appears, all existing memcpy() has to be rewritten to take advantage of it.
On the other hand, consider the CISC solution: REP MOVSB. A single two-byte instruction which the hardware decodes into whatever operations are most optimised for it. It handles updating the registers, the copy, and the loop using specialised hardware. It can transfer entire cache lines (64 bytes or more) per clock cycle. Software doesn't need to change to take advantage of e.g. a newer processor with a wider memory bus. It's tiny, so it uses next to no cache space, and once it's been fetched, decoded, and executing internally, the memory/cache buses are free for other purposes like transferring the data to be copied, or for the other cores to use. It's far easier for the CPU to decode a complex instruction internally into micro-ops and/or dispatch it to dedicated hardware than it is to try pattern-matching long sequences of simple instructions into a complex semantic unit for such hardware when it becomes available. It's hard enough for something as simple as memcpy(), never mind AES or SHA1.
- Why do compilers not strive to simplify their code-gen phase, or enable themselves to do advanced instruction-level program analysis, or both?
Compilers are complex because the CPUs they generate code for are also complex, and the CPUs are complex because of the reason above: this complexity is efficency. A compiler for a simple CPU could be simple, but that just means the CPU is so simple that there is nothing to optimise at the software level; hardly an ideal situation. I think it wouldn't be too hard to get GCC to generate only the subset of x86 instructions that most closely resembles MIPS, and compare the resulting binaries for size and speed. It should then be obvious why more complex instructions are good.
What is the minimum number of instructions a compiler could make use of to get everything done that it needs?
I came across an article that says 'mov is turing complete' [1]. But they had to do some convoluted tricks to use mov for all purposes.
I think it's safe to say that about 5-7 instructions are all that's needed to perform all computation tasks.
But then:
- Why do compilers not strive to simplify their code-gen phase, or enable themselves to do advanced instruction-level program analysis, or both?
- Why do microprocessors not strive for simplicity, implement only a handful of instructions in an optimized way, with a very small chip footprint, to be followed by proliferation of cores (think 256-core, 512-core, 1024-core).
Besides the completely valid reason that humans tend to overly-complicate their solutions, and then brag about it, the main reason is historical baggage and the need for backwards compatibility.
Intel started with a bad architecture design, and only made it worse decades after decades, by piling one bunch of instructions over another, and what we now have is a complete mess.
On the compiler front, the LLVM white-knights come along and tell people 'you guys are wimps for using C to do compilers. Real men use monsters like C++, with dragons like design-patterns. No one said compiler programming is supposed to be as simple as possible.'
To those lamenting javascript and the web being broken, wait till you lift the rug and get a peek at the innards of your computing platform and infrastructure!
[1] https://www.cl.cam.ac.uk/~sd601/papers/mov.pdf