Probably the #1 area that can produce results is avoiding/conquering the processor-memory gap: while processor performance has been growing exponentially, memory (bandwidth) performance has basically grown linearly. There is now a factor 1,000 difference between processor/memory speeds compared to 1980.

One of the areas that I have much hope for is near-data processing: since processors scale so much better, pretty much every peripheral device already has its own microcontroller. The idea behind NDP is basically to offload some data-heavy processing to the data layer. What if your disk layer could already preselect your data so the database wouldn't have to read and discard so many rows for each query? What if the network controller could evaluate your firewall rules itself, so dropped packets wouldn't have to interrupt the main CPU?

> What if your disk layer could already preselect your data so the database wouldn't have to read and discard so many rows for each query?

My impression is that that the process of filtering DB rows is sufficiently complex to need a full libc type execution environment. But taking a big step back in perspective, a famous example of filtering on processors connected to disks is Map-Reduce, aka Hadoop.

> What if the network controller could evaluate your firewall rules itself, so dropped packets wouldn't have to interrupt the main CPU?

Yes, this is a real thing: https://duckduckgo.com/?q=nic+packet+filter+offload

So NDP is essentially what Commodore did with their 1541 disk drive. That disk drive had a 6502 in there to complement the 6502 in the actual VIC-20.

From what I remember, the IBM System Z mainframes also do this sort of thing and have dedicated IO processors that can decode XML on the fly for you and other fun things like that.

As a matter of fact, I think this concept dates at least as far back as the IBM 360.

https://en.wikipedia.org/wiki/Channel_I/O

And you can find similar concepts in a standard PC today. The GPU is an example of offloading a workload to a specialized processor.

In addition, if each individual unit of memory hardware gained mini-CPU-like data processing capability, then you would have additional faux-CPU power that scaled linearly with memory; this would allow you to do some (embarassingly parallel) things much faster than just having one additional CPU per peripheral.

Every modern hard drive is a computer in its own right.

Seagate had a cool project where each hard drive ran linux and they used the physical sas cable to run a 2.5 Gbit network (or two actually) per drive.

So you could use that as block storage for luster, hadoop, or similar and enable things like direct disk to disk copies.

Cool idea, seems unlikely to hit a reasonable price point though.

This is first time I'm hearing it but NDP seems really cool. I like the idea of augmenting the memory and peripheral devices with intelligence.

Is Kryder's Law also dead? (2x memory every 13 months) If not, we could plan for petabyte local memory by 2030 and exabytes 2040s.

Making features smaller is far and away the most cost efficient way to make processors faster.

It is also relevant to remember fan size and processor design are at least two independent divisions in the same company (Intel, Samsung) and are often two separate companies (AMD, TSMC). Its not like the chip makers arent investing engineers in design.

I see. I wonder what there strategy would be like? To double the efforts in reducing fab size or in other factors.

We can of course do both, and there is a lot of work on improving the designs. But scaling down the transistor size is generally a wonderful thing: smaller transistors use less power, produce less heat, and you can fit way more of them in the same area (which enables things like larger caches, etc.)

Dennard scaling has been dead for a while, smaller transistors don't necessarily by default use less power anymore.

ok so I see that reducing fab size gives lot of benefits and they have steadily done so for past couple of decades. In doing so maybe they overlooked on design or other factors? And now that we've hit the wall on fab sizes (purportedly), maybe they can double their efforts on other factors.

That reminds me of this talk from 2013:

https://www.youtube.com/watch?v=JpgV6rCn5-g

The gist of it, as I remember it, is that radical design ideas were a bad investment while Moore's law ruled, because they were likely to be outperformed by simply shrinking the standard architecture; after Moore, design gains in importance, but don't expect anything like the performance improvements of the past half century.

Makes sense. Thanks for the video.

I think there really is quite a lot of innovation going on under the radar in the SoC space. Phones now have “neural engines” and all kinds of image-related processors for their cameras. Desktop CPUs come with integrated GPUs and there’s a lot of memory system consequences of that...

We are software limited also. AFAIK you could design a high level easy to use programming language with high performance vectorized array operations, but such languages at least are not in wide use. By high level I mean something like Ruby where you could specify lambdas for enumerables. But the vm would take care of vectorization and/or parallelization for you.