Communication overhead is the enemy of these highly parallel machines. The Connection Machines had many CPUs, but the unsung hero was the hypercube of connections between them [the clue is in the name]. According to Wikipedia it was a 12 dimension hypercube so every node had 12 high speed point-to-point I/O channels to adjacent nodes, which must have been a nightmare to implement and a nightmare to design software for. The cost of a CM-5 (Wikipedia again says $25 million) must have mostly been for this very specialised network.

It's hard to imagine this could have been competitive with a $25 million pile of beige PC boxes from the same era, but the PCs would have been starved of I/O (10 Mbps shared thick ethernet anyone?) so only applications which don't need much I/O between the nodes would be possible.

A "modern" CM-5 would ironically look much more like the pile of beige PCs, because it will have much less I/O -- these cheap chips only seem to have at most one or two fast channels (eg. ethernet and SDIO). There's no way to build these into a hypercube. It will be constantly limited by bandwidth and contention addressing other nodes in the cluster.

So I'd only build it for fun, not for practicality :-)

SpiNNaker[0] is a species of that, with multi-dimensional connections in a toroidal surface configuration. It's ARM-based, largely because the chief developer/project head is Steve Furber. Along with interviews concerning the BBC Micro and ARM, Computerphile did a video with Furber concerning SPiNNaker[1].

[0] https://en.wikipedia.org/wiki/SpiNNaker [1] https://www.youtube.com/watch?v=2e06C-yUwlc

Yes SpiNNaker looks very cool, also of course the BBC Micro connection as you say. I do wonder what the network architecture is, so now I'm going to have to watch that video you posted :-)

Edit: It's a toroid, which seems an unusual choice (because 2D) for something that's meant to simulate a brain. I wonder if a simple 3D cubic connection network would have been possible by adding more links between physically adjacent boards.

I'm surprised it was worthwhile to build, since the whole thing has only 7TB of ram. I guess the total memory bandwidth is really good.

I've noticed that a lot of those small arm chips are designed for external ram. I wonder if physically shared ram could overcome the I/O limitation.