Computational memory can't speed up 'everything' because most software doesn't benefit from single operations that span a large chunk of data (AKA SIMD ops, as on GPUs). This technique will always improve only a small select fraction of present and future software, like games/graphics, video and voice processing, and general purpose vector operations (like tensor ops in deep learning).

Or programs are not written to use computational memory because there isn't any for them to use.

Besides, computational memory is not restricted to SIMD. It's limited into local memory, but not at synchronized execution.

How many programs can even manage to use two full-featured cores in parallel, with everything made as easy as possible? How many even try to do calculations on a GPU?

I'm not holding my breath for more than 1% of programs to use this. Or if it gets built into GPUs, then 1% when possibly excluding rendering code.

How many programs need to? The ten year old laptop I'm writing this comment on is fast enough for everything I want to do. Even my Raspberry Pi is fast enough for almost everything I want to do.

There is no reason whatsoever that this would be limited to SIMD operations. They will be easy to do because you just spam a certain set of gates out in an array, but you could be laying down completely novel, or programatically determined, gates instead.

This idea has come around many times, all the way back to the ILLIAC IV, Thinking Machines, and the NCube. The only commercial success is GPUs, which really do have elements with both compute and memory crunching away in parallel. Other pure-crunch appilcations have been found for GPUs and things like them. There are now GPU-like compute units for servers, although in many cases they're still GPUs, with frame buffers and video hardware that will never generate an image.

I think one could make a case for NUMA systems being a dilution if this concept. But my recollection of the Illiac IV was pretty much what I’d call a NUMA system.

Brings to mind images of Conway's Game of Life, with computations moving about a grid of memory as they create and destroy new bits of data.

And that brings to mind http://www.conwaylife.com/w/index.php?title=HashLife, an example (of many) of an algorithmic improvement that gives a way higher speed up than parallellizing and throwing hardware at a problem.

Hashlife is a brilliant algorithm. For anyone not familiar (and vaguely interested in Conway's GOL), highly recommend reading up about it. I found it mildly mindblowing :)

Or Core War.

https://en.m.wikipedia.org/wiki/Core_War

It makes perfect sense, because this is already how good programmers look at the task, especially considering the penalties for memory access on secondary devices.