I think it is time to talk about new directions because we're still STUCK HARD in a rut begun in 1971 for general purpose compute. 4004 -> 8008 -> 8080 -> x86 and here we remain. Requirements and tech have both changed vastly since then. We've made some minor widening, into RISC, GPUs, and VLIW, plus we're just now looking into many cores, now that atomic physics is starting to push back on minimum feature sizes.

Two areas that have not been remotely explored are asynchronous design and massively parallel (think Connection Machine). In both cases, the compilers and tools were not up to the task, but perhaps we're ready to give both another try.

That's a rather closed view of history. Vector processors were the norm in supercomputers until the early 2000s, when they were killed off by general purpose CPUs like SPARC and POWER. Intel itself has made two big bets on alternative architectures--Itanium (essentially VLIW) and Xeon Phi (CMP gone insane), the former of which failed miserably, and the latter of which appears to be quietly dying.

One of the major problems with developing alternative highly-parallel architectures is that most algorithms, particularly as they are written in source code today, simply aren't parallelizable. There's only so much ILP that can be extracted, and data parallelism isn't achievable without very heavy annotation of source code. Getting high data parallelism (as GPGPU does) is sometimes possible with algorithmic rewrites, but you're talking about rewriting several portions of your code to make that happen. That's the reason NVidia is successful--they've got an offload model, so you only need to rewrite the kernels that need parallelism, and they've got the proprietary lock-in on CUDA since no one wants to rewrite that code several times. But even then, there are classes of code that can't be run on GPGPUs very well because you just don't have the necessary data parallelism.

> 4004 -> 8008 -> 8080 -> x86

The progression is more interesting than that: it's Datapoint 2200 -> 8008 -> 8080 -> x86. The Datapoint 2200 was a "programmable terminal" (really a desktop computer) that had a CPU made from TTL chips. They got Intel to make a MOS version of the processor, which was the 8008. (TI made a MOS processor for them too, beating Intel with the forgotten TMC 1795.) Because the Datapoint 2200 used serial shift-register memory, it needed to be little endian, and that's why the x86 is little endian.

Also, the marketing names make 8008 sound like an 8-bit version of the 4004, but they are really unrelated. The 4004, for instance, wasn't even von Neumann.

https://en.wikipedia.org/wiki/Datapoint_2200

I think that there have been discussions, but everyone being on this one-big-complex-cpu-doing-everything train over the years made companies pursue financial goals rather than security. You cannot have a complex and secure CPU. That would be too expensive, so it's not a question of will-it-work, rather will-it-sell.

Yeah, security is still not a profit feature.

I think it's more the humans that aren't up to the task, or at least have not built enough conceptual infrastructure to make it feasible to program these things.

A moderate number of people can grasp the imperative "do this then that" model, which CPUs strive so hard to maintain the appearance of despite speculative and out-of-order execution. Fewer people are comfortable with "functional" style programming. Other paradigms are relegated to corners. Multithreaded programming is still a bit of a black art.

Graphics programming only manages parallelism because there's an implicit huge map of "do this to every pixel" around everything. The actual shader programming language .. is imperative, because that's what people understand.

Yay for asynchronous :D

It's honestly criminal how little attention async gets.

But absolutely disagree on massively parallel processors. What do you think your GPUs, TPUs/DPUs not to mention Tilera types are?

GPU, TPU, DPU, etc are not general purpose compute: they are highly specialized for a certain tasks. Not familiar with Tilera, but it looks like up to 100 cores? That's not what I mean by massive either.

CM was massively parallel: 64,000 general purpose processors. I don't think we adequately explored its attributes. I bet we could make a decent emulation of a CM and its support tools now.

I think we need more architectures in wide use because a monoculture is bad for many reasons. One reason not so often mentioned is that it's then harder to distribute proprietary binaries.

If you make a living writing software, that sounds like an argument against more architectures.

Plenty of people make a living writing FOSS.

There are exceptions, but the vast majority of commercial software is not FOSS.

If anything, with a few shining examples, I've been totally underwhelmed by FOSS.

> the vast majority of commercial software is not FOSS.

The overwhelming majority of commercial software made is custom. Much of it is already Free (as well as bloody expensive), and the rest could be without changing anything to the underlying business models.

At the same time, the majority of commercial software used is proprietary. Because while Microsoft Word is only one piece of software, it is multiplied by the number of its users.

As a user, of course you'll find that almost all the software you would pay for is proprietary. But that's just availability bias: you miss all the software that has only one customer, and there is a lot of it.

The vast majority of software is not commercial software (as in, software that is sold. Most software is for internal use.)

> with a few shining examples, I've been totally underwhelmed by FOSS.

I suspect you think that most software is desktop software? This would be wrong.

Because governments don't back FOSS like they should.

Wait, what? Why should governments step in to back FOSS?

The city of Munich tried to switch to Linux. They just announced the plan to switch back to Windows - presumably they were underwhelmed too.

> Wait, what? Why should governments step in to back FOSS?

Because we all profit from it and there is little market incentive to back them. The government funds museums, mathematics and a lot of other things for less reasons compared to FOSS.

> The city of Munich tried to switch to Linux.

They did switch to a GNU/Linux system. It is hard to be different when everybody around you isn't. There were interoperability issues with other bodies. This is one more reason why the government needs to step in to not only fund FOSS but to legally require open formats. Also, FOSS needs to be taught in schools.

We don't all profit from it, it's a total distraction for most people.

I can't believe the amount of time I've wasted on Linux just trying to get my sound card to work, or a printer. Or I could just buy Windows.

FOSS has a lot of "me too" and "reinvent the wheel" as well, to the point of distraction. At this point I think it's borderline useless outside of compilers and scientific computing.

Also, I really wouldn't waste children's time on FOSS. "Here's how to spend a week on Stack Overflow fixing your printer" is not a core subject.

> We don't all profit from it, it's a total distraction for most people.

Collectively we do, just like we do from mathematics or museums. Also, imagine what the FOSS ecosystem could be if the underlying technologies where funded by governments around the world.

> I can't believe the amount of time I've wasted on Linux just trying to get my sound card to work, or a printer.

First, if these issues exist, then that is one more reason to fund it. Second, when did you try a modern distribution the last time? It has been a long time since I had problems like that. Usually, when I buy hardware and it doesn't work with free driver I send it back because it's non-functional in my eyes. It's very rare that I need to do that.

> Also, I really wouldn't waste children's time on FOSS. "Here's how to spend a week on Stack Overflow fixing your printer" is not a core subject.

Cleary, the hypothetical printer issue is not what should be tought. Good upstream support is the manufacturer problem, and regulation for labels of driver support would help a lot.

What should be thought are not specific programs but programming and the value of FOSS. Maybe, a few selected and curated and sufficiently funded FOSS programs can be presented to show how common problems could be solved.

I bet path dependence has something to say about that. Try to force people to use Dvorak keyboards at work. Even though that layout is demonstrably better (more comfortable and easier to learn), people are already used to Qwerty, still have a Qwerty keyboard at home, and would generally resist change.

Switching to GNU/Linux triggers similar problem: different GUI, still Windows at home, IT has to re-learn everything… Oh, and drivers. These aren't GNU/Linux's fault, but they still tend to underwhelm users.

I, mean, seriously: ever tried to have your grandma use LibreOffice? The number one problem won't come from bugs. It won't even come from its inability to read such and such Microsoft Word document. It will come from her inability to "save as .doc" or "save as .pdf" so other people can read the document (and even if she could, it's still a hassle). Network effects are not LibreOffice's fault, but they still tend to underwhelm users.

>We need heterogeneous systems.

If you are to develop a CPU with an ISA level security proof, you need a homogenous system.

Something like a simplest MCU with few registers and hardwired instructions, no pipeline or OOOE, and flat memory that can be 100% mathematically verificated.

The Transputer T800 was formally verified. It would also be a great candidate vor hardware isolation, with different processes running on different CPUs, linked via the Transputer's hardware channels.

https://www.cs.bris.ac.uk/~dave/transputer.html

UPDATE: T800 had 250K transistors, Skylake has 1-2 billion, so you could fit around 4K-8K T800s on the same kind of die.

So you are talking about a special purpose processor solely for the purpose of being 100% deterministic? It'll be slow, but potentially more secure than its general purpose counterparts. Which of both would be more reliable security-wise? If you put both in a single system respecting the purposes, a heterogeneous system is the result.

You will still get something very different from HSA's understanding of heterogeneous design or secure enclave processors (although those ones come closer to the idea).

The thing that is approaching it the most is a smartcard CPU. Flat X-i-P rom, cacheless, embedded mram or fram, <100 instructions, full register state determinism.

> let's not design and implement a hundred different architectures

...

> We need heterogeneous systems.

The first seems to be a call for fewer, the second for more architectures. Which is it?

Rather less the same system architecture reiterated in different cpu architecture implementations (AMD, Intel), and more special purpose processors for a heterogeneous/hybrid system architecture following certain standards.

> The whole Von-Neumann system architecture is way too old for today's problems.

Why?

Amdahl's law. The von Neumann bottleneck continues to get bigger and bigger with each iteration. We can do far more calculations in the same amount of time on a chip, due to the increasing number of transistors we can fit on them, and also improvements in the internal designs (superscalar architectures, etc). The speed of memory access however, has only been increasing at marginal rates in comparison, and now we have increasing contention for execution units in the processor to access it as we increase the number of distinct cores. Caching, prefetching, etc. have partially helped to mitigate some of the problems, but only as far as you can avoid cache misses (common in object oriented code, due to frequent pointer dereferencing). We're still at the stage, with all these technologies, that the VNB is what's preventing us from doing more in the same amount of time. Writing high-performance code is also difficult as often the hardware specializations are inaccessible to a programmer, and he has to rely on the compiler/microassembler to do the right thing. Often this means designing code in ways to suit the CPU (eg, vectorizing all your data), rather than to suit the programmers (eg, OOP).

We should really be aiming towards NUMA, many-core processors with high-speed message passing between cores, and a programming model which suits development over that kind of architecture (eg, the Actor model).

Because instructions and data share the same memory space in general? Potentially, in such a system architecture any bug or bad implementation might lead to memory leaks from the whole memory range.