We are nowhere near an atom well into 2005 at least, but things were getting pipelined and there was a ton of SRAM on each die, there were improvements to fetch pipelines (well, SPECTRE and friends really), everything was happening as fast as it could with out-of-order pipelines, with prefetch and every other speedup out there including auto-vec.
The "how big is an atom?" is sort of post-2010 thinking, at least the way I saw it happen.
Modems was one place where this sort of thing I saw personally happen - 9600 baud modems to 56kbps happened so fast and it almost looked it would keep going that way (Edge was 115kbps, now I can't even do a page load over it) with DSL suddenly dropping in to keep the same copper moving even faster again.
For example, these techniques improve instructions per clock, at the cost of adding transistors:
* pipelining (late 80’s early 90’s) MIPS R3000, Intel 486, Motorola 68040
* superscalarity (early 90’s) Pentium, DEC Alpha 21064, MIPS R8000, Sun SuperSparc, HP PA-7100
* out of order execution (mid 90’s) Intel Pentium Pro and later, DEC Alpha 21264, Sun UltraSparc, HP PA-8000
* SIMD (vector) instructions (mid/late 90’s) Pentium MMX (integer) Pentium III (floating point), DEC Alpha 21264
* multithreading (early 2000’s) Pentium 4, planned chips from DEC and MIPS
We also got improved branch predictors and larger, more-associative caches.
But it feels like most of that progress stopped in the early 2000’s, and the only progress is slapping more cores on a die. I mean, if you can put 8 cores in a consumer level CPU, you have 8 times as many transistor (give or take) as you need to implement a CPU. Nobody seems to be building a higher IPC CPU with 2x transistors, even though they clearly have the transistors to do it.
> Nobody seems to be building a higher IPC CPU with 2x transistors
I mean, there are designs like this, but they run into problems of cache invalidation and internal bus contention.
The way to get around this is to enforce rules about how much can be shared between cores, i.e. make the NUMA cores not present a UMA abstraction to the developer but rather be truly NUMA, with each core having its own partition of main memory.
But then you lose backward compatibility with... basically everything. (You could run Erlang pretty well, but that’s basically it.)
Sure, but that's not helping the general case. Only specific types of workloads. You could argue that adding lots of special-purpose hardware doesn't hurt from a transistor count (we have plenty) or power perspective (turn them off when not needed), but it can make layout tricky and reduce clock speed (which slows down everything else).
> [...] cache invalidation and internal bus contention [...] NUMA
Sure, but the context from spamizbad was specifically single-core performance. (I probably opened up a can of worms by mentioning hypertheading). The problem is many real-world workloads that add business value are not embarrassingly parallel problems. If it worked like that, Thinking Machines would have swept the court since the 1980's (they had NUMA like you are describing).
The point is that, since about 2005-2010ish, single-thread performance has mostly stalled. Intel CPUs can issue slightly more instructions per clock. AMD has a slightly better branch predictor. But performance growth has mostly been the result of adding more cores (Except Apple's M1 has some magic).
The things I previously mentioned gave big IPC gains on a diverse set of real workloads. Some innovations, like pipelining and multi-issue were responsible for 2x-4x IPC each. Pipelining, in particular, was a trick that also helped clock speeds.
All those innovations happened between the late 1980's and early 2000's. So an observer during that time might have just assumed that similar innovations would keep coming. But they haven't. A Pentium III has probably around 15x IPC compared to an i386 (maybe 60x if you include SIMD), in addition to a 40x higher clock speed (some of which came from adding more transistors).
How can you add transistors (say 2x or 3x) to a CPU to double performance on diverse, real-world problems that don't parallelize well? My point is, I don't think anyone knows, so it is irrelevant whether there is a physical limit to transistor shrinkage. We don't even know what to do with the transistors we have, so who cares if we can't have more?
Yes, but by itself that isn't a single core vs. multicore issue. It's a process node question generally.
That said, Intel in particular did try to push single-core frequency at least a beat too far. They demoed a 10GHz prototype at one IDF--and other companies were mocking them for it.
The story that a very senior Intel exec gave me at the time was something to the effect that, of course, they knew that they were going to have to go multi-core but Microsoft was really pushing back because they weren't confident in multicore desktop performance in particular.
I don't get that. Moore's law is about transistor size shrinking, but that has an obvious end — transistors can't be smaller than one atom across. I feel like "everyone" did know that, even in the 90s. Or, at least, it was mentioned in every explanation of Moore's law itself.
> Low/no code drag-and-drop style programming tools would take away many dev jobs
I mean, they did, but they also created consultant jobs to replace the ones they took away. See: Salesforce.
The need to write source code can be taken away; but the need to do requirements analysis and deliberate architectural design cannot. So you end up with people who do 80% of a programmers' job, except for the coding part.