I heard somewhere that a major problem with the PS3 cell was it had a very limited local memory for its programs and data, which meant the host (main) cpu had to constantly moved memory back and forth from main memory to individual cell memory.

Yes.

The idea of the Cell architecture is that programs were supposed to be organized as an assembly line, with data flowing into a processor in a stream, and coming out the other side after processing. That's a good fit to audio processing.

Each of the seven support processors had only 256KB (not MB) of local RAM, plus a wide long-latency path to the 256MB main memory, the other support processors, and the PowerPC CPU.

That 256KB was the problem. Not enough for a frame. Not enough for a level. Not enough for AAA title games.

That's the problem with the lots of little processors approach. If all the little guys are memory-starved, you can't get much done, except on very regular problems. Finite element analysis and neural nets are the big wins for such architectures. They're not general purpose.

Note that GPUs aren't arrays of isolated CPUs. All units work against the same memory, mostly. They're closer to shared memory multiprocessors.

> However, with AI today, it's completely possible that it changes the economics of being able to write software which can fully utilize new and novel architectures affordably.

I'm not complete sold on that one. You will need training material. While there have been impressive results even for architectures/programming languages for which I would not have expected to be much training material to be available, there will be a minimum. I'm afraid for a new architecture unlike everything else, your AI coding assistant won't be of much help.

But models can just ingest large parts of the documentation into their context window, write the code based on that (which they'd likely be surprisingly good at, adapting to the entirely different memory mechanics and so on using just a handful examples) and test+iterate on it until it produces the expected results (which they can compare against a regular implementation) and performs well.

> However, with AI today, it's completely possible that it changes the economics of being able to write software which can fully utilize new and novel architectures affordably.

With AI today, it's completely possible that it can't change the economics of being able to write software which can fully utilize new and novel architectures affordably.

So essentially it's gambling $30M on a coin toss that a clanker can magically fix something that hasn't worked the last 200 times we tried it.