People keep bringing up the Mill processor in response to parallelization; I've been hearing about it for ~5 years now. And yet, where is it? I see lots of talk about its architecture, but there's no hard comparisons on things like "we get DGEMM at 2x the FLOPS/W of NVidia's latest GPU" or "we get 2x SPECCPU 2017 as Intel's latest processor."

There's a saying that you should always work to minimize the number of axes along which you're innovating at the same time. Every innovation carries the risk that you might be wrong and taking too many gambles guarantees that one will blow up in your face. This is complicated by the fact that sometimes you can't make progress by changing just one thing, there are some local maximums you can only get out of by changing multiple things at once.

The Mill has some sets of innovations that seem to come together, for instance the Spiller, Belt, and memory speculation model seem hard to seperate. But I'm pretty sure you could detach that from having a virtually addressed cache and their protection model. And then there's stuff like the super weird organization structure they have. I feel like they're setting themselves up for failure by trying to make changes in so many ways all at once.

Based on their simulations, they are estimating a much better power / performance ratio than existing superscalar designs.

However, starting up an entirely new computer architecture isn't easy. They want to retain control of their company, so they aren't accepting tons of VC money. Progress is slow.

That's because they don't have a real product, and they likely never will. For the most part, the ISA just needs to be 'good enough' to allow things like branch prediction, speculative execution, out-of-order execution, etc. You can do that with x86, ARM, POWER, whatever - the instruction set doesn't matter a whole lot, it's just how many/what type of transistors you can throw at it, and how hard you want to work on timing and power optimization.

That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish.

> That's because they don't have a real product, and they likely never will.

That was sort of my point; however cool the conceptual architecture is, they've definitely reached the point where they ought to be able to get at least rough comparisons to competitive hardware. That such comparisons are lacking should be taken as a sign that the architecture is too immature to make those claims. I just wish people would stop bringing it up as the magic solution to parallelism, since there's insufficient evidence for those claims.

> That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish.

I've been swayed by the idea that the future of processor technology is essentially accelerator land. You're going to have a few big CPU cores because they're good for code that has no real parallelization potential, but you're going to also have clusters of accelerators that are optimized for different kinds of workloads (such as data-heavy, communication-light workloads that are good for GPGPU).

> That's because they don't have a real product, and they likely never will.

That is possible, their progress has been slow.

> For the most part, the ISA just needs to be 'good enough' to allow things like branch prediction, speculative execution, out-of-order execution, etc. You can do that with x86, ARM, POWER, whatever - the instruction set doesn't matter a whole lot, it's just how many/what type of transistors you can throw at it, and how hard you want to work on timing and power optimization.

If you watch all their videos, then this isn't really correct.

Architecture matters a lot. With modern processors, there is a vast gulf between the instruction set used (the programming model) and how the processor actually works. A lot of the die area on the chip (aside from caches) isn't devoted to actual computation (like adding numbers in an ALU) but moving data around and scheduling.

The Mill Computing approach is to move all this functionality back out of the hardware into the software compilation / loading step. Where it can be done once and more efficiently. They can then use that die area to add in more functional units (because the architecture is wide-issue) and more cache.

There is only so much that clever architecture can accomplish, but that is still an order of magnitude more than what is being done now with existing architectures, so this seems a worthy pursuit.

> The Mill Computing approach is to move all this functionality back out of the hardware into the software compilation / loading step.

Why will they succeed where Itanium failed? There's a long, long history where people expect compiler smarts to maximize performance of hardware, and compilers just aren't up to it. It's also very necessary to demonstrate this on larger, more complex code, rather than focusing on things like GEMM kernels.

There's a lot that needs to be justified, and the Mill people just haven't presented the justification.

> Why will they succeed where Itanium failed? There's a long, long history where people expect compiler smarts to maximize performance of hardware, and compilers just aren't up to it. It's also very necessary to demonstrate this on larger, more complex code, rather than focusing on things like GEMM kernels.

Ivan Godard mentions Itanium a few times during the talks he has given. So they are at least aware of that architecture and its faults.

I guess it all comes back to the question of where all this instruction scheduling ought to be. Yes, it is being done in hardware now, because that is the path that offers software compatibility at the binary encoding level.

But in a sensible world, is that really where the instruction scheduling should be? And if it can be done in hardware, I don't see any theoretical reason it can't be done in software.

The question, then, becomes how to do this in a practical manner. It remains to be seen if the Mill Computing scheme will be successful. I would argue that we (as an industry) haven't failed enough with the software scheduling approach.

> But in a sensible world, is that really where the instruction scheduling should be? And if it can be done in hardware, I don't see any theoretical reason it can't be done in software.

Hardware instructions can take a variable amount of time to complete, depending on their inputs--particularly branches and memory instructions. That means that the set of operations that are ready to execute is a fundamentally dynamic property, and software can only make a static decision.

The branch predictor should mean that in most cases the latency of branches should be constant. And if its wrong that's just as much a problem for an OoO machine as it is for the Mill.

For memory accesses yes, that's a big problem which is why there's so much effort in the design devoted to making software memory speculation feasible. The Itanium put some effort into that too but generally at a much higher overhead and the Mill's solution for dealing with memory faults in speculated accesses was frankly brilliant.

> Hardware instructions can take a variable amount of time to complete, depending on their inputs--particularly branches and memory instructions. That means that the set of operations that are ready to execute is a fundamentally dynamic property, and software can only make a static decision.

Everything on the Mill CPU architecture is statically scheduled, because everything has a fixed latency.

Loads / stores from L1 cache are also a fixed latency. If you're going out to DRAM, then you'll have to wait a lot no matter what architecture. Branches are also fixed latency, assuming the target has been prefetched.

So then it is a question of how good is the prefetching, and they've got some techniques to improve that too. They've also got techniques to help with cache pressure and effectively double the I-cache size without the usual power and latency penalty.

Details are in the papers and videos, if you are interested.

Radically different architectures can help with some problems via specalization as seen in the TPU. I think we'll see more of that.

Sure, but my point was that specialization is basically a one-time trick. Once you build a 5nm (or whatever they call the process node, as the numbers are basically marketing these days) TPU, GPU, CPU, what have you, that's kind of it. You could spend a ton of effort on clever architecture and physical design, and maybe get a 2nd generation with a ~10-20% performance boost (but maybe not even better power efficiency, just more power). If you haven't maxed out your area/power budget, you might be able to scale up a bit more with a bigger, power hungrier die or MCM/interposer, but GPUs have basically already played that card.

So each 'problem' that's worth it might get a specialized ASIC or two, but all of that could very well run its course in the next 5-10 years, and things look kind of bleak for performance gains after that.