I'm not so sure. CPUs are stupidly fast, now - the bottleneck is virtually always the time it takes to move data in and out of memory. What if, instead of trying to bring the data to the calculation, you brought the calculation to the data? You could speed up everything.

Fun to think about: in this scenario, the main job of an "optimizing compiler" would be to organize things spatially so that, as a calculation moved around in memory, it always found itself right next to the data it needed, when it needed it.

Computational memory can't speed up 'everything' because most software doesn't benefit from single operations that span a large chunk of data (AKA SIMD ops, as on GPUs). This technique will always improve only a small select fraction of present and future software, like games/graphics, video and voice processing, and general purpose vector operations (like tensor ops in deep learning).

Or programs are not written to use computational memory because there isn't any for them to use.

Besides, computational memory is not restricted to SIMD. It's limited into local memory, but not at synchronized execution.

How many programs can even manage to use two full-featured cores in parallel, with everything made as easy as possible? How many even try to do calculations on a GPU?

I'm not holding my breath for more than 1% of programs to use this. Or if it gets built into GPUs, then 1% when possibly excluding rendering code.

How many programs need to? The ten year old laptop I'm writing this comment on is fast enough for everything I want to do. Even my Raspberry Pi is fast enough for almost everything I want to do.

There is no reason whatsoever that this would be limited to SIMD operations. They will be easy to do because you just spam a certain set of gates out in an array, but you could be laying down completely novel, or programatically determined, gates instead.

This idea has come around many times, all the way back to the ILLIAC IV, Thinking Machines, and the NCube. The only commercial success is GPUs, which really do have elements with both compute and memory crunching away in parallel. Other pure-crunch appilcations have been found for GPUs and things like them. There are now GPU-like compute units for servers, although in many cases they're still GPUs, with frame buffers and video hardware that will never generate an image.

I think one could make a case for NUMA systems being a dilution if this concept. But my recollection of the Illiac IV was pretty much what I’d call a NUMA system.

Brings to mind images of Conway's Game of Life, with computations moving about a grid of memory as they create and destroy new bits of data.

And that brings to mind http://www.conwaylife.com/w/index.php?title=HashLife, an example (of many) of an algorithmic improvement that gives a way higher speed up than parallellizing and throwing hardware at a problem.

Hashlife is a brilliant algorithm. For anyone not familiar (and vaguely interested in Conway's GOL), highly recommend reading up about it. I found it mildly mindblowing :)

Or Core War.

https://en.m.wikipedia.org/wiki/Core_War

It makes perfect sense, because this is already how good programmers look at the task, especially considering the penalties for memory access on secondary devices.

AI stuff would be easy to implement on it, but general computation will almost certainly benefit just as much once research is done. It's a fundamentally different architecture when your assembly language can include instructions that essentially re-wire the FPGA which is your processor/storage. That makes possible entirely new classes of algorithms.

The researchers are those focused in AI applications and as such I expect them to focus advertising that. This technology has far more potential than just that, right down into simple stuff like realtime waveform or spectrographic analysis. Being able to store and process on the exact same 'silicon' without having to run out a few inches to retrieve data will boost latency-sensitive applications.