I'm not a chip designer or anything like that, but I think it isn't the difficulty of creating processors with so many cores (or arrays of processors equivalent) that has stopped us, but the cost benefit of increasing cores to the expense. I use an HPC cluster at work, but for what I'm doing it makes sense to use the hundreds of cores available. Indeed, most of it is embarrassingly parallel.

Aside from GPUs, we have progressed on cores, just not as fast as we thought, and multi-core hasn't been around for all that long. Multi-chip modules like AMD's chiplet approach were needed to get things like 64-core Threadrippers. I can't imagine trying to get 64 full CPU cores onto a single die at an acceptable yield.

That's kind of what GPUs became, wasn't it? They do have thousands of cores, just not in the mega range

Strictly speaking, GPU's have thousands of ALU's/execution units, spread over a few dozen full cores at most. And reaching full utilization of that amount of compute is not always easy either, many problems just don't parallelize that widely.

In GPU we can start talking about kilo cores.