News for hackers. Stuff that matters.
Re:ARM Cortex-A72 fetch and branch processing by GregarianChild on Monday December 14, 2020 @04:58PM (#25423940)

   Why do and then un-do?
The reason is that we've not yet found a better way of making fast general purpose processors.

The OOO (= out-of-order) approach with high-quality prediction (e.g. branch, value) to processor micro-architecture makes sense if you have to mask a lot of memory-access latency, which comes from data-dependent (= unpredictable) memory access. General purpose workloads have a lot of that. (If memory access patterns are more predictable, you'd probably run your workload on a GPU or TPU or DSP, or some other accelerator.) Compilers, whether ahead-of-time, or JIT, have not got enough information to schedule commands in a way that can mask memory latency the way an OOO scheduler inside a processor can. Stalling the pipeline because you are waiting for data to arrive from memory is disastrous for performance and to be avoided at all cost. Intel's Itanium was based on the premise that it is possible statically to schedule well enough, but that has been considered a failure for general workloads. Moreover, if you let a compiler schedule you need to have ISA extensions that allow you to communicate scheduling information to the processor which is not cost free (for example you may spend precious bits to encode scheduling order that you then cannot use for other things).

I suspect that the very last word has not been spoken in this discussion, but everything obvious to make dataflow competitive (and quite a lot more) has been tried, and failed.

Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @05:31PM (#25424266)
IMO a lot of the problem is simply that powerful players (eg. Intel) had a vested interest in tying things to the platform, and the megaliths we've built on top all assume this construction, so even if there are clear ways in which things can be done better (there are), you'd have to reinvent the world to do them. Fundamental innovation is really difficult, and OoO CPUs avoid the need for it.
Re:ARM Cortex-A72 fetch and branch processing by cpgxiii on Monday December 14, 2020 @07:33PM (#25425460)
It's not at all clear that better ways exist for general purpose code. A lot of time and effort has been spent by major players, with very little to success to show for it.

Intel, quite infamously, tried to make "something better" work with several generations of Itanium and failed impressively against the reality that a sufficiently smart compiler cannot be made to work. You can't blame any of the big players for holding back - Intel, MS, and HP were "all in" on Itanium as the 64-bit future - if it could have been made to work, it would have been widely adopted.

At the same time, Transmeta was building a X86-on-VLIW, which, guess what, also failed to be a competitive product. Nvidia, of course, decided to pick up from where Transmeta left off to build what ended up being an aarch64-on-VLIW design. In both cases, the hope was that the low-level translation would allow the benefits of a VLIW architecture while avoiding the "sufficiently smart compiler is impossible" problem, and in both cases performance just wasn't good enough to be competitive.

Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @07:40PM (#25425530)
> A lot of time and effort has been spent by major players

This really isn't the case IMO. Itanium is one attempt 20 years ago, it didn't try to rewrite the world (compatibility with standard C code is a huge crutch; GPUs couldn't exist in that environment for example), and the design sucked. And for 20 years since it has been sufficient to invoke the name to scare away any attempt of actually trying something new, even when they don't make the same mistakes, or even work anything like Itanium.

> Transmeta

It's in the name: the most significant part of the design was all the effort spent on compatibility with the dominant paradigm, running stock x86 code! That hugely limits what you can do.

Re:ARM Cortex-A72 fetch and branch processing by cpgxiii on Monday December 14, 2020 @08:04PM (#25425731)
Itanium was not one attempt, it was several generations of trying to make the concept work. And it didn't.

> And for 20 years since it has been sufficient to invoke the name to scare away any attempt of actually trying something new, even when they don't make the same mistakes, or even work anything like Itanium.

People have been invoking Itanium for the last 20 years because that's what you do with expensive, painful lessons. Just like "branch delay slots are bad" (MIPS et al) or "no sub-word-size-access is bad" (Alpha) or "there's such a thing as too-relaxed memory ordering" (Alpha) or "putting too much magic in the ISA is bad" (iAPX 432), some lessons don't need to be repeatedly revisited and repeated.

I get that the Mill team wants to prove everyone wrong, but the only way to show that is working hardware and that doesn't exist (and given the time they've been working, probably won't ever).

> compatibility with standard C code is a huge crutch; GPUs couldn't exist in that environment for example

Except, guess what, Cuda and OpenCL look exactly like C in many ways. The major features they don't support (well) are dynamic memory allocations and branchy code - good luck trying to make an architecture work that can't efficiently handle malloc. People have tried a lot of different language+hardware architectures over the years, and all of them have been outperformed by OoO machines.

> compatibility with the dominant paradigm, running stock x86 code

If either Transmeta or Nvidia could have made a standout VLIW architecture, they would have had plenty of business. Defense and HPC customers have absolutely no problem with esoteric architectures so long as the performance is good enough. You could build the craziest architecture, and if you can show it'll be a winner, the DoE will buy it.

Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @08:16PM (#25425824)
> Itanium was not one attempt

It was one ISA, and a terrible one at that, which is my point.

> Except, guess what, Cuda and OpenCL look exactly like C in many ways.

You're missing my point, which is about rewriting the world. C's syntax is not the problem.

Re:ARM Cortex-A72 fetch and branch processing by cpgxiii on Monday December 14, 2020 @09:46PM (#25426400)
> It was one ISA, and a terrible one at that

Except, you know, the whole point of VLIW designs was that by offloading complexity to the compiler and runtime you'd be less limited by hardware design decisions and be able to improve with time. Provide real examples of how Itanium could have been done better.

> You're missing my point, which is about rewriting the world

No, I'm not. "Compatibility with standard C code" means memory allocations and branches. That's it. Most "real" OSs need an MMU, but there are a few research systems that don't.

You should also keep in mind that lots of different models have been tried. Compiled machine code? Done. JIT-recompiled machine code? Done. GC'd Lisp on hardware? Done. GC'd ada on hardware? Done. GC'd JVM on hardware? Done. Binary translation to "better" hardware? Done, lots of different times. "Rewrite the world" has been tried a fair number of times before, and it didn't win out.

You're also totally missing the point that several architectures, x86/x86_64 most notably, have succeeded because ISA and hardware implementation aren't locked together tightly. Processor designers and users are not nearly as constrained together as you believe.

Re:ARM Cortex-A72 fetch and branch processing by renox on Tuesday December 15, 2020 @04:04AM (#25428416)
> it didn't win out.

It didn't win because 1) making a top performing CPU is very expensive 2) you can only only recoup your investment by selling a lot of CPUs. So you need to be very efficient with the installed base of software..

While in the past this favored x86 / x86-64 compatibility, this is shifting towards ARM compatibility.. Maybe this will favour experimentations? ARM ISA is far less complex to implement so from a technical POV this could help companies trying to experiment.

Re:ARM Cortex-A72 fetch and branch processing by cpgxiii on Tuesday December 15, 2020 @08:22AM (#25430033)
> So you need to be very efficient with the installed base of software

That's only somewhat true. Until the modern era of cloud hyperscalers, fairly weird architectures with poor compatibility were quite successful, especially in HPC and defense markets. Cell and its derivatives were successful for their time. Any number of quirky microcontroller architectures were successful before the Cortex-M family came to dominate (and some still are). Yes, the startup costs are always going to be high, but success in the broader market isn't necessary for a performant architecture to survive.

> Maybe this will favour experimentations?

I don't think so, if anything the high performance aarch64 designs like Apple's M1 are more similar to x86_64 designs than previous generations. Large caches, wide execution, serious OoO, stronger-than-expected memory ordering. I think more experimentation will come from the RISC-V community targeting more specific applications.

Re:ARM Cortex-A72 fetch and branch processing by renox on Tuesday December 15, 2020 @09:15AM (#25430476)
>if anything the high performance aarch64 designs like Apple's M1 are more similar to x86_64 designs than previous generations.

And? If an 'exotic ISA' can really offer some improvement, it must be able to compete with 'the best of the best' implementation of current ISA..

Re:ARM Cortex-A72 fetch and branch processing by cpgxiii on Tuesday December 15, 2020 @09:58AM (#25430856)
The point is that if you're looking for distinctly different approaches than the currently dominant x86_64, aarch64 implementations are not where to look.
Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @10:29PM (#25426637)
> Except, you know, the whole point of VLIW

It's really indicative of the problem I'm talking about that so many people think the options are either a standard, Intel Core-style OoO or Itanium-style VLIW. Not only is VLIW a massive range of potential things, and Itanium is not the natural implementation of it, but I never even said we should go with VLIWs! I'm personally more enamoured with CG-OoO style hierarchical reordering, though were rewriting the world possible I'd do something that exposed more of that to software (probably microthread style).

> Provide real examples of how Itanium could have been done better.

Although Itanium seems like a dead-end strategy to me, three things it got really wrong were:

* software data misprediction recovery,

* overly large register files,

* compiler target locks in uarch decisions.

(Please mind that it's been a while since I looked at the Itanium ISA.)

The Mill is sort of what Itanium would be with all the rough edges sanded over, though it's sort of a moot point since the Mill will never exist outside of talks.

Again, static schedules aren't the approach I'd advocate for, since there are some things they just can't do.

Re:ARM Cortex-A72 fetch and branch processing by pjmlp on Tuesday December 15, 2020 @02:02AM (#25427775)
CUDA was redesigned to work like C++ actually.
Re:ARM Cortex-A72 fetch and branch processing by 1996 on Monday December 14, 2020 @05:47PM (#25424405)
> you'd have to reinvent the world to do them

It reminds me of how MIPS branching is wasted on nops:

https://electronics.stackexchange.com/questions/28444/mips-p...

"The trick in writing efficient code is to put in an instruction that will be useful as part of the loop that is being taken executed, but do no harm if the branch is not taken.

The MIPS designers were counting on compiler writers to write clever enough code generators to handle this efficiently. However many do not (including Microchips C32 compiler, based on GCC), and just put NOP's after every branch, wasting both code space and cycles"

Re:ARM Cortex-A72 fetch and branch processing by ekiwi on Monday December 14, 2020 @06:49PM (#25425061)
The problem with delay slots is that their optimal size is somewhat tied to the particular micro-architecture. I.e., different MIPS CPU implementations would have different optimal branch delay slot sizes. This doesn't matter too much if your compiler is allowed to optimize for your particular implementation, but we are still mostly shipping binaries and thus you would have to make trade-offs for portability reasons.
Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @08:18PM (#25425835)
They're also pointless once you do even a small amount of dynamic local reordering, which every fast CPU does.
Re:ARM Cortex-A72 fetch and branch processing by gpderetta on Monday December 14, 2020 @05:42PM (#25424363)
GPUs have been extremely successful, so have DSPs, each on their niche. They just don't work well on branchy pointer chasey code. Intel itself tried to move beyond OoO and utterly failed.

There is no conspiracy here.

Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @06:38PM (#25424946)
GPUs were only able to come about into the mainstream because they were brought about and developed alongside their use-case. Much the same can be said for camera ISPs or neural accelerators. GPUs only started tackling more general workloads long after they were a standard, and even today GPGPU is a bit of a mess.

If GPUs didn't come around when they did, and games were all CPU-based today, you wouldn't have an easy time inventing and selling a GPU.

> Intel itself tried to move beyond OoO and utterly failed.

One attempt in 2001 (and a few before then when the market was more competitive but before people had the context we have today) by the company with market power doesn't invalidate my point as much as reinforce it. People have tried approximately nothing in the last 20 years, all pointing back to this one bad design Intel did 20 years ago as if that proves the taken path is the only valid path.

Re:ARM Cortex-A72 fetch and branch processing by MaxBarraclough on Monday December 14, 2020 @07:01PM (#25425162)
> If GPUs didn't come around when they did, and games were all CPU-based today, you wouldn't have an easy time inventing and selling a GPU.

I'm not convinced. You'd just need to show that hardware acceleration can produce worthwhile gains for computer graphics, which doesn't strike me as insurmountable. Perhaps start with an FPGA.

> People have tried approximately nothing in the last 20 years

Do you follow the 'Mill' architecture? https://en.wikipedia.org/wiki/Mill_architecture

Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @07:11PM (#25425241)
Good luck showing that when all graphics code is designed to be run on CPUs, nobody has spent more than trivial effort making GPU-compatible graphics algorithms, you can't afford new nodes, your GPU is first-generation (and so far from optimal), you can only use CPU memories (who would invent HBM if GPUs didn't exist?), you can only do fixed-function logic efficiently, you have no demos that don't look like programmer art, and you have no way of getting past the adoption hurdle.

Even realtime RT, which is mostly just a compatible add-on to raster, only managed to get off the ground when NVIDIA decided to go all-in and tank the criticism for a generation by effectively forcing it on the customer. No other company could have done that, even though Imagination was pushing for RT hardware for ages before NVIDIA.

> Do you follow the 'Mill' architecture?

Yes. https://medium.com/@veedrac/to-reinvent-the-processor-671139...

Re:ARM Cortex-A72 fetch and branch processing by gpderetta on Monday December 14, 2020 @07:44PM (#25425568)
To be constructive, I do believe there is a future beyond classic OoO, but dynamic scheduling is going to be part of it, although possibly not of instructions but of BBs or even microthreads. The papers you referencing are quire interesting as they validate some ideas I was idly thinking about. I still would call these designs OoO though, just extended.

The problem is that building a CPU is very expensive and high risk, going even a bit beyond the classic OoO can bankrupt you. Intel tried with Itanic and AMD almost ate its lunch; AMD tried with buldozer and shared execution units and almost collapsed; SUN tried with flock of chickens and is no more; Nvidia tried with denver, and well, I guess they had a lot of cash. Both AMD and Apple just made completely orthodox designs and were complete successes.

Evolution is going to be in small steps, unless a new design can prove to be 10x better than the existing ones which is very unlikely.

Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @08:05PM (#25425739)
> I do believe there is a future beyond classic OoO, but dynamic scheduling is going to be part of it, although possibly not of instructions but of BBs or even microthreads.

I agree, I find it hard to see how you could abandon dynamic scheduling without unreasonable sacrifices.

My point isn't to say otherwise, just that we see less innovation than we would otherwise were our ecosystem built to handle it. GPUs have benefited hugely from the ability to rapidly change the ISA and a lot of internal details between each generation, and for each competitor to have their own take and extensions (Apple even has its own API, to support its TBDR features). If we had that sort of freedom in CPUs, I'm sure we'd have something a lot better than four variants of practically-the-same-thing by four different companies.

Re:ARM Cortex-A72 fetch and branch processing by brandmeyer on Monday December 14, 2020 @08:41PM (#25426012)
> Nvidia tried with denver, and well, I guess they had a lot of cash

More likely: They recognized the failure and cut the program before losing too much money.

Re:ARM Cortex-A72 fetch and branch processing by gpderetta on Tuesday December 15, 2020 @04:56AM (#25428684)
True, also Denver wasn't bad, just not good enough.
Re:ARM Cortex-A72 fetch and branch processing by my123 on Tuesday December 15, 2020 @05:47AM (#25429014)
Denver was superseded by the Carmel architecture. (which is sold on Tegra Xavier today)

The NVIDIA CPU team is alive and well.

Re:ARM Cortex-A72 fetch and branch processing by brandmeyer on Wednesday December 16, 2020 @05:03PM (#25449409)
There's a big difference between cancelling a project and firing the team. I haven't seen any evidence that Carmel utilizes binary translation.
Re:ARM Cortex-A72 fetch and branch processing by cpgxiii on Monday December 14, 2020 @07:45PM (#25425577)
The reality is that the embarrassingly parallel nature of graphics has been long understood, as well as the advantages of having graphics be handled by a different, dedicated processor (see Intel's i860 in the early 1990s).

Unless in some alternate universe frequency scaling and memory bandwidth were never an issue, you're always going to come back to the reality that a parallel processor optimized for minimally-branchy math will be more efficient at graphics tasks. The early days of fixed-function hardware are, if anything, an aberration.

Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @08:11PM (#25425795)
And yet most non-games graphics workloads in industry were CPU rendered until 2010ish, I think. The draw of defaults is extremely strong.
Re:ARM Cortex-A72 fetch and branch processing by cpgxiii on Monday December 14, 2020 @09:25PM (#25426296)
No. Back to the very first graphical workstations, people have people putting dedicated processors in computers for graphics (ex Apollo Computer using the TMS320 DSP for graphics math). The entire reason for legacy decisions in OpenGL is from non-gaming users, namely CAD vendors, who have been doing this for a long time.
Re:ARM Cortex-A72 fetch and branch processing by Veedrac on Monday December 14, 2020 @09:43PM (#25426389)
The context is very different if you're talking about the early 1980s. I recognize (and have said before in this thread) that my arguments about lock-in don't apply to then.
Re:ARM Cortex-A72 fetch and branch processing by pjmlp on Tuesday December 15, 2020 @04:00AM (#25428388)
And the benefit is that now you have companies like OTOY doing basically CUDA based real time rendering.
Re:ARM Cortex-A72 fetch and branch processing by gpderetta on Monday December 14, 2020 @07:28PM (#25425401)
> GPUs only started tackling more general workloads long after they were a standard, and even today GPGPU is a bit of a mess.

yes, because other workloads are niche and until very recently didn't warrant investing on dedicated architectures for them.

Re:ARM Cortex-A72 fetch and branch processing by rsynnott on Monday December 14, 2020 @06:30PM (#25424858)
Intel _tried_ a major departure: https://en.wikipedia.org/wiki/Itanium

Spoiler: It didn't end well.

Re:ARM Cortex-A72 fetch and branch processing by pjmlp on Tuesday December 15, 2020 @04:02AM (#25428401)
Only because AMD exists and had access to the patents that allowed them to come up with an alternative.

Had it not been the case and Itanium would have been pushed until it eventually replaced x86.

Re:ARM Cortex-A72 fetch and branch processing by rsynnott on Tuesday December 15, 2020 @07:07AM (#25429505)
I think that's very unlikely, really; Itanium just really wasn't great. To a large extent, if AMD64 hadn't existed it would have been necessary to create it (either that or Intel could've gone off in a RISC-y direction and made something along the lines of a DEC Alpha, I suppose). But in any case, it's hard to see an alternate universe where Itanium was actually successful.
Re:ARM Cortex-A72 fetch and branch processing by pjmlp on Tuesday December 15, 2020 @08:01AM (#25429854)
Windows and HP-UX were already available on Itanium before amd64 came into scene.

Without access to Intel patents who would come with enough money and ecosystem support to actually be a challenge to Itanium?

Re:ARM Cortex-A72 fetch and branch processing by fulafel on Tuesday December 15, 2020 @01:05AM (#25427492)
What we call general purpouse processors are just a model that has coevolved with software. If we got better at making software for parallel processors that are different from current GP CPUs, we could call them those processors purpouse too.