But at some point making your table larger just defeats the purpose of the library itself. Your table becomes the new library, and you have to walk around on it and look up things in these piles of books. So you make a smaller table in the middle of the big table.
Your fundamental limitation is how small you can make a memory cell, not how big you want to make a cache. That’s akin to making the books smaller print size so you can fit more on the same size table.
The other physical aspect we’re dealing with is propagation delay and physical distance. That’s where the library analogy really shines: if there’s a minimum size to a book and a minimum size of you (the person doing the research) this corresponds roughly to minimum cell sizes and minimum wire pitch, so you’re ultimately limited in the density you can fit within a given volume.
Once you have enough registers, having more mean lowers active utilization for any given instruction (bad use of space, vs. fast pipelined access to cached stack) or higher levels of parallel instruction dispatch (much greater complexity, and even greater inefficiency for branching misses).
Then you have to update instruction sets, which could be impossible given how tightly they fit in current instruction sizes.
Ergo, increasing register banks is a major architecture & platform change from hardware to software redesign, with heavy end user impact, and a fair chance of decreasing performance.
In contrast, anything that improves caching performance is a big non-disruptive win.
Then think of registers as just part of the fetch & store pipelines for staging operations on stack values.
Forth goes all in with this approach.
If you add more registers, the cost per register increases rapidly, and you very quickly hit your limits.
If you make registers wider, that's still very expensive, and you introduce extra steps to get to your data most of the time.
So no, you can't do that in a reasonable way.
2.5d is just too easy and effective - we're going to have lots more chiplets, and only the cool ones will get stacked.
remember, we're talking TB/s these days.
That is just mainstream reporting.
If one actually went and read the paper referred or what the context was. It was always the same thing. It was all about the economics, all the way back from early 90s. We cant do x node because it would be too expensive to sustain it at a node every two years.
Smartphone era ( Referring to Post iPhone launch ) essentially meant we ship an additional ~2 Billions Pocket computer every year including Tablet. That is 5x the most optimistic projection to traditional PC model at 400M / year. ( Which we never reached ). And that is ignoring the Server market, Network Market, GPU market, AI Market etc. In terms of transistor and revenue or profits the whole TAM ( Total Addressable Market ) went up at least 10x more than those projection. Which is essentially what scale us from 22nm to now 3nm, and all the way to 2nm and 1.4nm. And my projection of 1nm by 2030 as well. I even wrote on HN in ~2015 I have a hard time to see how we could sustain this post 3nm. At the time when trillion dollar company was thought to be impossible.
On the other side of things, the cost projection to next node ( e.g 2nm ), and next next node (e.g 1.4nm ) was always higher than what its turns out. As with any large project management it is was better to ask and project more in case shit hits the fan. ( Intel 10nm ) But every time TSMC has executed so well.
So as you can see there is a projection mismatch at both ends. Which is why the clear sign of progress coming to end keeps being wrong.
> and can expect 1000X improvement in the years to come at least.
I just want to state that this figure keeps being throw around. It was Jim Keller comparing at the time Intel 14nm ( Which is somewhere close to TSMC N10 ) to hypothetical physics limit. At 3nm we are at least 4x pass that. Depending on how you want to measure it we could reach less than 100x by 2030.
AI trend could carries us forward to may be 2035. But we dont have another product category like iPhone. Server at hyperscaler are already at a scale growth is slowing. We will again need to substantially lower the development cost of leading node ( My bet is on the AI / Software side ) and some product that continues to grow the TAM. May be Autonomous Vehicles will finally be a thing by 2030s ? ( I doubt it but just throwing in some ides ).
They are mostly programmed in managed languages, where the respective runtimes and OS collaborate, in order to distribute the computing across all available cores in the best way possible, with little intervention required from the developers side.
Additionally, the OS frameworks and language runtimes collaborate in the best way to take advantage of each specific set of CPU capabilities in an almost transparent way.
Quite different from the regular POSIX and Win32 applications coded in C and C++, where everything needs to be explicitly taken care of, which is what kind of prevents most of the cool CPU approaches to take off, sitting there idle most of the time.
I was under the impression that distributing workloads across many CPU cores (or HW threads) is done at the process and thread level by the OS? That gives managed and unmanaged languages the same benefits.
Managed languages provide higher level primitives that makes it easier to create a multi-threaded application. But isn't that still manually coded in the mainstream managed languages?
I'm thinking of inherently CPU-intensive custom workloads. UI rendering and IO operations become automatically distributed with little intervention.
Or am I missing something, where there is "little intervention required from the developers side" to create multi-threaded apps?
So the schedulers can decide in a more transparent way what runs where, specially on Android side, where the on-device JIT/AOT compilers are part of the loop.
Additionally, there is more effort on having the toolchains explore SIMD capabilities, where on C and C++ level one is expected to write that code explicilty.
Yes, auto-vectorization isn't as good as writing the code explicitly, however the latter implies that only a niche set of developers actually care to write any of it.
Hence why frameworks like Accelerate exist, even if a JIT isn't part of the picture, the framework takes the best path depending on available hardware.
Likewise higher level managed frameworks offer a better distribution between the parallel processing taking part across CPU, GPU or NPU, which again on classical UNIX/Win32 in C and C++, have to be explicility programmed for.
Such higher level frameworks can of course also be provided in such languages, e.g. CUDA and SYCL, howver then we start discussing about programmer culture to adopt such kind of tooling in classical LOB applications.
I don't know these, but from a quick googling it still looks like explicit multi-threading? Albeit with higher level primitives than in older languages, but still explicit?
> auto-vectorization
I'm not sure I see a hard dividing line between older languages and managed ones as far as auto-vectorization? Sure, a higher-level language might make it easier for the compiler since it knows more about potential side effects, but simple and local C code doesn't have any side effects either.
> with little intervention required from the developers side
> Hence why frameworks like Accelerate exist, even if a JIT isn't part of the picture
Accelerate looks nice, but it still looks like it has to be called explicitly in the user code?
> Likewise higher level managed frameworks offer a better distribution between the parallel processing taking part across CPU, GPU or NPU, which again on classical UNIX/Win32 in C and C++, have to be explicility programmed for.
I'm not sure I understand, can you give more explicit examples?
My point here isn't that managed languages don't give big benefits over C. I prefer Python and C# when those can be used.
It's more that I don't see "automatic parallel processing" as a solved problem?
Sure, we get better and better primitives for multi-threading, and there are more and more high-level parallel libraries like you mentioned. But for most cases, the programmer still has to explicitly design the application to take advantage of multiple cores.
Totally agree.
> AI trend could carries us forward to may be 2035. But we dont have another product category like iPhone.
There will be fancier iPhones with on board offline Large Language Models and other Foundation Models to talk to, solving all kinds of tasks for you that would require a human assistant today.
TSMC or ASML? Or both? I am not trying to be dismissive, just curious about who deserves the credits here.
Remember,
1. TSMC didn't beat Intel because they had the newer EUV first. They beat Intel before the whole thing started.
2. If having EUV Machines from ASML were enough, Samsung would have been the 2nd given they are or they will use it for NAND and DRAM. And yet they are barely competing.
3. It is not like Global Foundry dropped off racing the leading edge node for no reason.
4. TSMC has always managed to work around any road block when ASML failed to deliver their promise on time.
5. Quoting from CEO of ASML, half jokingly but also half true "Dont ask us about how those EUV machine are doing. Ask TSMC, they know that thing better than we do."
Of course there is large number of small companies around the whole Pure Play Foundry business in which TSMC ex-CEO calls it the Grand Alliance. You need every party to perform well for it to happen. This is somewhat different to Samsung and Intel, both are ( more or less ) much more vertically integrated.
I remember a quote from Von Braun where he learned to use the word 'impossible' with the greatest caution.
When you have a significant fraction of the GDP of a super power dedicated to achieving some crazy engineering task, it almost certainly can be done. And I wouldn’t bet against our hanger for better chips.
Every now and then Moore’s law hits a roadblock. Some experts see that as a clear sign that it’s reaching its end. Others that it’s already dead, because actually, the price per transistor has increased. Others that it’s physics, we can approach Y but after X nm it can’t be done.
Then you read others that claim that Intel has just been lazy enjoying its almost monopoly for the past decade and was caught off guard by TSMC’s ultraviolet prowess. Or people who really know how the sausage is made, like Jim Keller, enthusiastically stating that we are nowhere near any major fundamental limitation and can expect 1000X improvement in the years to come at least.
Anyway, it’s really fun to watch, like I said. Hard to think of a field with such rollercoaster-like forecasting while still delivering unparalleled growth in such a steady state for decades.