> Something tells me Nvidia made next to no gains for this generation.

Sounds to me like they made "massive gains". In the end, what matters to gamers is

1. Do my games look good? 2. Do my games run well?

If I can go from 45 FPS to 120 FPS and the quality is still there, I don't care if it's because of frame generation and neural upscaling and so on. I'm not going to be upset that it's not lovingly rasterized pixel by pixel if I'm getting the same results (or better, in some cases) from DLSS.

To say that Nvidia made no gains this generation makes no sense when they've apparently figured out how to deliver better results to users for less money.

I use DLSS type tech, but you lose a lot of fine details with it. Far away text looks blurry, textures aren’t as rich, and lines between individual models lose their sharpness.

Also, if you’re spending $2000 for a toy you are allowed to have high standards.

Sure, but compute is a limiting factor.

DLSS might not be as good as pure unlimited pathtracing, but for a given budget it might be better than rasterization alone.

I’m saying that it’s different enough that you shouldn’t compare the two.

If we're moving towards real time tracing compute is going to always be a limitting factor, as it was in the days of pre rendering. Granted currently raster techniques can simulate ray trace pretty well in many scenarios and looks much better in motion, IMO that's more limitation of real time ray trace. There's a bunch of image quality improvements beyond raster to be gained if enough compute is throw at ray tracing, i think a lot of dlss / frame generation goal is basically to offload more cpu to generate higher IQ hero frames while filling in blanks.

8k traditional rendering at 144Hz is a lot of pixels. We are seeing a 25%/3 years improvement cycle on traditional rendering at this point, and we need about 8x improvement in current performance to get there.

2040 is definitely possible, but certainly not guaranteed.

Makes you wonder how far ahead displays will be in 2040. I can imagine display prices falling in price and increasing in quality to the point where many homes just have displays paneled around the walls instead of paint.

Unfortunately, there are no 4K displays with 200+ DPI on the market. If you want high DPI you either need pick glossy 5k@27" or go to 6k/8k.

a 27" 1080p screen has 37ppd at 2 feet.

a 42" 4k screen has 51ppd at 2 feet.

a 27" 8k screen has 147ppd at 2 feet which is just absurd.

You have to get to 6 inches for the PPD to be 61

I know I'll be gunning for the 42" 8K's whenever they actually reach a decent market price. Sigh, still too many years away.

I cannot brag with sharp eyesight, but I can definitely tell difference between 4k@27" at 60cm = 73PPD and 5k@27" at 60cm = 97PPD. Text is much crisper on the latter.

I've also compared Dell 8k to 6k. There is a still a difference, but it is not that big.

You must have exceptional eyesight.

There are 4k 24" monitors (almost 200 DPI) and 4k 18.5" portable monitors (more than 200 DPI) you can buy nowadays

Cause it ain't about the gameplay or the usefulness. It's all about that big dick energy.

Making better looking individual frames and benchmarks for worse gameplay experiences is an old tradition for these GPU makers.

A game running at 60 fps averages around ~16 ms and good human reaction times don’t go much below 200ms.

Users who “notice” individual frames are usually noticing when a single frame is lagging for the length of several frames at the average rate. They aren’t noticing anything within the span of an average frame lifetime

if you added 200ms latency to your mouse inputs, you’d throw your computer out the of the window pretty quickly.

30 FPS is 33.33333 MS 60 FPS is 16.66666 MS 90 FPS is 11.11111 MS 120 FPS is 8.333333 MS 140 FPS is 7.142857 MS 144 FPS is 6.944444 MS 180 FPS is 5.555555 MS 240 FPS is 4.166666 MS

Going from 30fps to 120fps is 25ms which is totally 100% noticeable even for layman (I actually tested this with my girlfriend, she could tell between 60fps and 120fps as well), but these generated frames from DLSS don't help with this latency _at all_.

Although the nVidia Reflex technology can help with this kind of latency in some situations in some non quantifiable ways.

That's just it isn't. This stuff isn't "only detectable by profession competitive gamers" like many are proposing. It's instantly noticeable to the average gamer.

Though some CRT emulation techniques require more than that to scale realistic 'flickering' effects.

Just moving my mouse around, I can tell the difference between 60 and 144 fps when I move my pointer from my main monitor (144 hz) to my second monitor (60 hz).

Watching text scroll is noticeably smoother and with less eye tracking motion blur at 144 hz versus 60.

An object moving across my screen at 144 fps will travel fewer pixels per frame than 60 fps. This gain in motion fluidity is noticeable.

... now I wonder ... Do DLSS models take mouse movements and keypresses into account?

"MPEG uses motion vectors to efficiently compress video data by identifying and describing the movement of objects between frames, allowing the encoder to predict pixel values in the current frame based on information from previous frames, significantly reducing the amount of data needed to represent the video sequence"

By the way, I thought these AI things served to increase resolution, not frame rate. Why doesn't it work that way?

- Data Center: Third-quarter revenue was a record $30.8 billion

- Gaming and AI PC: Third-quarter Gaming revenue was $3.3 billion

If the gains are for only 10% of your customers, I would put this closer to the "next to no gains" rather than the "massive gains".

i d like to point you to r/FuckTAA

>Do my games run well

if the internal logic is still in sub 120 hz and it is a twichy game, then no

https://www.youtube.com/watch?v=ghT7G_9xyDU

we do see power requirements on the high end parts every generation, but that may be to maintain the desired SKU price points. there's clearly some major perf/watt improvements if you zoom out. idk how much is arch vs node, but they have plenty of room to dissipate more power over bigger dies if needed for the high end.

People who keep giving intel endless shit are probably very young and don't remember how innovative Intel was in the 90s and 00s. USB, PCI-Express, Thunderbolt, etc., all Intel inventions, plus involvement in Wifi and wireless telecom standards. They are guilty of anti competitive practices and complacency in the last years but their innovations weren't just node shrinks.

Intel’s strategy after it adopted EM64T (Intel’s NIH syndrome name for amd64) from AMD could be summarized as “increase realizable parallelism through more transistors and add more CISC instructions to do key work loads faster”. AVX512 was that strategy’s zenith and it was a disaster for them since they had to cut clock speeds when AVX-512 operations ran while AMD was able to implement them without any apparent loss in clock speed.

You might consider the more recent introduction of E cores to be an innovation, but that was a copy of ARM’s big.little concept. The motivation was not so much to save power as it was for ARM but to try to get more parallelism out of fewer transistors since their process advantage was gone and the AVX-512 fiasco had showed that they needed a new strategy to stay competitive. Unfortunately for Intel, it was not enough to keep them competitive.

Interestingly, leaks from Intel indicate that Intel had a new innovation in development called Royal Core, but Pat Gelsinger cancelled it last year before he “resigned”. The cancellation reportedly lead to Intel’s Oregon design team resigning.

AMD up until zen 5 didn't have a full AVX-512 support so not exactly a fair comparison. Intel designs don't suffer from that issue AFAIU for couple of iterations already.

But I agree with you, I always thought and I still do that Intel has a very strong CPU core design but where AMD changed the name of the game IMHO is the LLC cache design. Hitting as much as ~twice lower LLC latency is insane. To hide this big of a difference in latency, Intel has to pack larger L2+LLC cache sizes.

Since LLC+CCX design scales so well AMD is also able to pack ~50% more cores per die, something Intel can't achieve even with the latest Granite Rapids design.

These two reasons let alone are big things for data center workloads so I really wonder how Intel is going to battle that.

https://en.wikipedia.org/wiki/AVX-512#CPUs_with_AVX-512

It also took either 4 or 6 years for Intel to fix its downclocking issues, depending on whether you count Rocket Lake as fixing a problem that started in enterprise CPUs, or require Sapphire Rapids to have been released to consider the problem fixed:

https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Dow...

  (1) drive 2x AVX-512 computations
  (2) handle 2x AVX-512 memory loads + 1x AVX-512 memory store

The latter makes a big impact wrt available memory BW per core, at least when it comes to the workloads whose data is readily available in L0 cache. Intel in these experiments is crushing AMD by a large factor simply because their memory controller design is able to sustain 2x64B loads + 1x64B stores in the same clock. E.g. 642 GB/s (Golden Cove) vs 334 GB/s (zen4) - this is a big difference and this is something that Intel had for ~10 years whereas AMD was able to solve this with zen5, basically only with the end of 2024.

Former one limits the theoretical FLOPS/core capabilities since single AVX-512 FMA operation in zen4 is implemented as two AVX2 uops occupying both FMA slots per clock. This is also big and, again, this is something where Intel had a lead up until zen5.

Wrt downclocking issues, they had a substantial impact with Skylake implementation but with Ice Lake this was a solved issue and this was in 2019. I'm cool with having ~97% of max freq budget available with heavy AVX-512 workloads.

OTOH AMD is also very thin with this sort of information and some experiments show that turbo boost clock frequency on zen4 lowers from one CCD to another CCD [1]. It seems like zen5 exhibits similar behavior [2].

So, although AMD is displaying continuous innovation for the past several years this is only because they had a lot to improve. Their pre-zen (2017) designs were basically crap and could not compete with Intel who OTOH had a very strong CPU design for decades.

I think that the biggest difference in CPU core design really is in the memory controller - this is something Intel will need to find an answer to since AMD matched all the Intel strengths that it was lacking with zen5.

[1] https://chipsandcheese.com/p/amds-zen-4-part-3-system-level-...

[2] https://chipsandcheese.com/p/amds-ryzen-9950x-zen-5-on-deskt...

https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi...

https://www.ixpug.org/images/docs/ISC23/McCalpin_SPR_BW_limi...

Your 642 GB/s figure should be for a single Golden Cove core, and it should only take 3 Golden Cove cores to saturate the 1.6 TB/sec HBM2e in Xeon Max, yet internal bottlenecks prevented 56 Golden Cove cores from reaching the 642 GB/s read bandwidth you predicted a single core could reach when measured. Peak read bandwidth was 590 GB/sec when all 56 cores were reading.

According to the slides, peak read bandwidth for a single Golden Cove core in the sapphire rapids CPU that they tested is theoretically 23.6GB/sec and was measured at 22GB/sec.

Chips and Cheese did read bandwidth measurements on a non-HBM2e version of sapphire rapids:

https://chipsandcheese.com/p/a-peek-at-sapphire-rapids

They do not give an exact figure for multithreaded L3 cache bandwidth, but looking at their chart, it is around what TACC measured for HBM2e. For single threaded reads, it is about 32 GB/sec from L3 cache, which is not much better than it was for reads from HBM2e and is presumably the effect of lower latencies for L3 cache. The Chips and Cheese chart also shows that Sapphire Rapids reaches around 450 GB/sec single threaded read bandwidth for L1 cache. That is also significantly below your 642 GB/sec prediction.

The 450 GB/sec bandwidth out of L1 cache is likely a side effect of the low latency L1 accesses, which is the real purpose of L1 cache. Reaching that level of bandwidth out of L1 cache is not likely to be very useful, since bandwidth limited operations will operate on far bigger amounts of memory than fit in cache, especially L1 cache. When L1 cache bandwidth does count, the speed boost will last a maximum of about 180ns, which is negligible.

What bandwidth CPU cores should be able to get based on loads/stores per clock and what bandwidth they actually get are rarely ever in agreement. The difference is often called the Von Neumann bottleneck.

Correct.

> That is also significantly below your 642 GB/sec prediction.

Not exactly the prediction. It's an extract from one of the Chips and Cheese articles. In particular, the one that covers the architectural details of Golden Cove core and not Sapphire Rapids core. See https://chipsandcheese.com/p/popping-the-hood-on-golden-cove

From that article, their experiment shows that Golden Cove core was able to sustain 642 GB/s in L1 cache with AVX-512.

> They do not give an exact figure for multithreaded L3 cache bandwidth,

They quite literally do - it's in the graph in "Multi-threaded Bandwidth" section. 32-core Xeon Platinum 8480 instance was able to sustain 534 GB/s from L3 cache.

> The Chips and Cheese chart also shows that Sapphire Rapids reaches around 450 GB/sec single threaded read bandwidth for L1 cache.

If you look closely into my comment you're referring to you will see that I explicitly referred to Golden Cove core and not to the Sapphire Rapids core. I am not being pedantic here but they're actually different things.

And yes, Sapphire Rapids reach 450 GB/s in L1 for AVX-512 workloads. But SPR core is also clocked @3.8Ghz which is much much less than what the Golden Cove core is clocked at - @5.2GHz. And this is where the difference of ~200 GB/s comes from.

> Reaching that level of bandwidth out of L1 cache is not likely to be very useful, since bandwidth limited operations will operate on far bigger amounts of memory than fit in cache, especially L1 cache

With that said, both Intel and AMD are limited by the system memory bandwidth and both are somewhere in the range of ~100ns per memory access. The actual BW value will depend on the number of cores per chip but the BW is roughly the same since it heavily depends on the DDR interface and speed.

Does that mean that both Intel and AMD are basically of the same compute capabilities for workloads that do not fit into CPU cache?

And AMD just spent 7 years of their engineering efforts to implement what now looks like a superior CPU cache design and vectorized (SIMD) execution capabilities only to be applicable very few (mostly unimportant in grand scheme of things) workloads that actually fit into the CPU cache?

I'm not sure I follow this reasoning but if true then AMD and Intel have nothing to compete against each other since by the logic of CPU caches being limited in applicability, their designs are equally good for the most $$$ workloads.

I am in disagreement with that obviously.

I said that supporting 2 AVX-512 reads per cycle instead of 1 AVX-512 read per cycle does not actually matter very much for performance. You decided that means I said that AVX-512 does not matter. These are very different things.

If you try to use 2 AVX-512 reads per cycle for some workload (e.g. checksumming, GEMV, memcpy, etcetera), then you are going to be memory bandwidth bound such that the code will run no faster than if it did 1 AVX-512 read per cycle. I have written SIMD accelerated code for CPUs and the CPU being able to issue 2 SIMD reads per cycle would make zero difference for performance in all cases where I would want to use it. The only way 2 AVX-512 reads per cycle would be useful would be if system memory could keep up, but it cannot.

I also agree "some" (for example low-arithmetic-intensity) workloads will not benefit from a second L1 read port.

But surely there are other workloads, right? If I want to issue one FMA per cycle, streaming from two arrays, doesn't that require maintaining two loads per cycle?

  * Some time passes
  * Load 1 finishes
  * Some time passes
  * Load 2 finishes
  * FMA executes

There is an inherent assumption in this that the loads will finish serially rather than in parallel, and it would seem reasonable to think that the loads will finish in parallel. However, in reality, the loads will finish serially. This is because the hardware is serial. On the 9800X3D, the physical lines connecting the memory to the CPU can only send 128-bits at a time (well, 128-bits that matter for this reasoning; we are ignoring things like transparent ECC that are not relevant for our reasoning). An AVX-512 load needs to wait for 4x 128-bits to be sent over those lines. The result is that even if you issue two AVX-512 reads in a single cycle, one will always finish first and you will still need to wait for the second one.

I realize I did not address L2 cache and L3 cache, but much like system RAM, neither of those will keep up with 2 AVX-512 loads per cycle (or 1 for that matter), so what will happen when things are in L2 or L3 cache will be similar to what happens when loads come from system memory although with less time spent waiting.

It could be that you will end up with the loop finishing a few cycles faster with the 2 AVX-512 read per cycle version (because it could make the memory prefetcher realize the linear access pattern a few cycles faster), but if your loop takes 1 billion cycles to execute, you are not going to notice a savings of a few cycles, which is why I think being able to issue 2 AVX-512 loads instead of 1 in a single cycle does not matter very much.

Does my explanation make sense?

Let's look at a more interesting case. We have a dataset bigger than L3. We touch a small part of it with one kernel. That is now in L1. Next we do a second kernel where each of the loads of this part are L1 hits. With two L1 ports, the latter is now twice as fast.

Even better, we can work on larger parts of the data such that it still fits in L2. Now, we're going to do the above for each L1-sized piece of the L2. Sure, the initial load from L2 isn't happening as fast as 2x64 bytes per cycle. But still, there are many L1 hits and I'm measuring effective FMA throughput that is _50 times_ as high as the memory bandwidth would allow when only streaming from memory. It's simply a matter of arranging for reuse to be possible, which admittedly does not work with single-pass algorithms like a checksum.

Do you find this reasoning convincing?

I am also not sure what CPU this is. On recent AMD processors at the very least, it should be impossible to get FMA throughput that is 50 times higher from L1 cache bandwidth than system memory bandwidth. On the Ryzen 7 9800X3D for example, a single core is limited to about 64GB/sec. 50 times more would be 3.2TB/sec, which is ~5 times faster than possible to load from L1 cache even with 2 AVX-512 loads per cycle.

I wonder if you are describing some sort of GEMM routine, which is a place where 50 times more FMA throughput is possible if you do things in a clever way. GEMM is somewhat weird, since without copying to force things into L1 cache, it does not run at full speed, and memory bandwidth from RAM is always below peak memory bandwidth, even without the memcpy() trick to force things into L1 cache. That excludes the case where you stuff GEMV in GEMM, where it does become memory bandwidth bound.

I ask because you say that the results are similar to Zen4 so this would sorta imply that you run and measure single-core implementation? Intel in multi-core load-store looses a lot of bandwidth when compared to Zen3/4/5 since there's a lot of contention going on due to Intel cache architecture.

They do not. The chip has 105MB L3 cache and they tested on 128MB of memory. This exceeds the size of L3 cache and thus, it is not a proper test of L3 cache.

> If you look closely into my comment you're referring to you will see that I explicitly referred to Golden Cove core and not to the Sapphire Rapids core. I am not being pedantic here but they're actually different things.

Sapphire Rapids uses Golden Cove cores.

> And yes, Sapphire Rapids reach 450 GB/s in L1 for AVX-512 workloads. But SPR core is also clocked @3.8Ghz which is much much less than what the Golden Cove core is clocked at - @5.2GHz. And this is where the difference of ~200 GB/s comes from.

This would explain the discrepancy between your calculation and the L1 cache performance, although being able to get that level of bandwidth only out of L1 cache is not very useful for the reasons I stated.

> I'm not sure I follow this reasoning but if true then AMD and Intel have nothing to compete against each other since by the logic of CPU caches being limited in applicability, their designs are equally good for the most $$$ workloads.

Your reasoning was that the cores hypothetically being able to load/store AVX-512 faster was important for performance. The reality is that this does not really matter because of bottlenecks in the memory hierarchy. That is why Zen 4 did extremely well on AVX-512 code without the ability to do as many concurrent loads and stores.

Intel and AMD primarily compete in compute bound scenarios where memory bandwidth does not matter. This is why the vast amount of work they do on each CPU generation is designed to increase the amount of compute done per cycle on average (e.g. improved branch predictors, larger reorder buffers, wider cores, decreased memory access times from larger caches and better prefetch, etcetera). There do exist memory bound workloads (e.g. memset, memcpy, checksumming, GEMV, BLAS levels 1 and 2 in general...), but the memory bandwidth from L1 cache rarely matters for them since their working sets are often far too big. While there are still some memory bound operations done on CPUs, they are a small portion of what the CPUs do and the cases where they are not a small portion have long been moved to GPUs (e.g. inference, which primarily uses GEMV for token generation) since GPUs dwarf CPUs in memory bandwidth by 1-2 orders of magnitude.

L1 cache exists for latency and its high memory bandwidth is a side effect of that. Getting high memory bandwidth out of L1 cache does not matter as far as competitiveness goes since working sets are far bigger than the 80KB of L1 cache in Golden Cove. If working sets actually fit in L1 cache for its memory bandwidth to matter, there would be no need for the L2 cache and no need for the huge L3 cache (although perhaps a small cache shared by all cores could make sense). People would not bother building/buying machines with gigabytes or even terabytes of RAM either.

First, you claimed that there was no L3 BW test. Now, I am not even sure if you're trolling me or lacking knowledge or what at this point?

Please do tell what you consider a "proper test of L3 cache"? And why do you consider their test invalid?

I am curious because triggering 32 physical core threads to run over 32 independent chunks of data (totaling 3G and not 128M) seems like a pretty valid read BW experiment to me.

> Sapphire Rapids uses Golden Cove cores.

Right, but you missed the part that former is configured for the server market and the latter for the client market. Two different things, two different chips, different memory controllers if you wish. That's why you cannot compare one to each other directly without caveats.

Chips and Cheese are actually guilty of doing that but it's because they're lacking more HW to compare against. So some figures that you find in their articles can be misleading if you are not aware of it.

> You seem to view CPU performance as being determined by memory bandwidth rather than computational ability.

But that's what you said trying to refute the fact why Intel was in a lead over AMD up until zen5? You're claiming that AVX-512 workloads and load-store BW are largely irrelevant because CPUs are anyway bottlenecked by the system memory bandwidth.

> That would be true if the primary workload of CPUs were memory bandwidth bound workloads, but it is not since the primary workloads of CPUs is compute bound workloads. Thus, how fast CPUs read from memory does not really matter for CPU workloads.

I am all ears to hear what datacenter workloads you have in mind that are CPU-bound?

Any workload besides the most simplest one is at some point bound by the memory BW.

> The purpose of a CPU’s cache is to reduce the von Neumann bottleneck by cutting memory access latency.

> That way the CPU core spends less time waiting before it can use the data and it can move on to a subsequent calculation.

> How much memory throughput CPUs get from L1 cache is irrelevant to CPU performance outside of exceptional circumstances.

You're contradicting your own claims by saying that cache is there to hide (cut) the latency but then you continue to say that this is irrelevant. Not sure what else to say here.

> but they are truly exceptional since any importan workload where memory bandwidth matters is offloaded to a GPU because a GPU often has 1 to 2 orders of magnitude more memory bandwidth than a CPU.

99% of the datacenter machines are not attached to the GPU. Does that mean that 99% of datacenter workloads are not "truly exceptional" for whatever the definition of that formulation and they are therefore mostly CPU bound?

Or do you think they might be memory-bound but are missing out for not being offloaded to the GPU?

I claimed that they did not provide figures for L3 cache bandwidth. They did not.

> Now, I am not even sure if you're trolling me or lacking knowledge or what at this point?

You really should be grateful that a professional is taking time out of his day to explain things that you clearly do not understand.

> Please do tell what you consider a "proper test of L3 cache"? And why do you consider their test invalid?

You cannot measure L3 cache performance by measuring the bandwidth on a region of memory larger than the L3 cache. What they did is a partially cached test and it does not necessarily reflect the true L3 cache performance.

> I am curious because triggering 32 physical core threads to run over 32 independent chunks of data (totaling 3G and not 128M) seems like a pretty valid read BW experiment to me.

This does not test L3 cache bandwidth at all. This would be a generic memory bandwidth test. Their graphs showing performance at different amounts of memory is designed to show the performance of the memory hierarchy. The moment you exceed the amount of cache at a certain level, the performance transitions to different level. They did benchmarks on different amounts of memory to get the points in their graph and connected them.

> Right, but you missed the part that former is configured for the server market and the latter for the client market. Two different things, two different chips, different memory controllers if you wish. That's why you cannot compare one to each other directly without caveats.

The Xeon Max chips with its HBM2e memory is the one place where 2 AVX-512 loads per cycle could be expected to be useful, but due to internal bottlenecks they are not.

Also, for what it is worth, Intel treats AVX-512 as a server only feature these days, so if you are talking about Intel CPUs and AVX-512, you are talking about servers.

> But that's what you said trying to refute the fact why Intel was in a lead over AMD up until zen5? You're claiming that AVX-512 workloads and load-store BW are largely irrelevant because CPUs are anyway bottlenecked by the system memory bandwidth.

I never claimed AVX-512 workloads were irrelevant. I claimed doing more than 1 load per cycle on AVX-512 was irrelevant because you are going to be bottlenecked on the first load operation for anything that is memory bandwidth bound. Intel losing its lead to AMD is due to entirely different reasons than AVX-512 loads per cycle. This is obvious when you consider that most workloads do not touch AVX-512.

To be clear, when you can use AVX-512, it is useful, but the ability to do 2 loads per cycle does not add to the usefulness very much.

> I am all ears to hear what datacenter workloads you have in mind that are CPU-bound?

Note that I said compute bound, not CPU bound. Nearly all datacenter workloads run on CPUs are compute bound. Memory bandwidth bound things are better run on GPUs, which have far more memory bandwidth.

> Any workload besides the most simplest one is at some point bound by the memory BW.

This is wrong. It does not matter how many experts you consult. Every single one will try to explain the von Neumann bottleneck to you upon hearing this.

> You're contradicting your own claims by saying that cache is there to hide (cut) the latency but then you continue to say that this is irrelevant. Not sure what else to say here.

There is no contradiction. The cache is there to hide latency. It is largely not there to provide bandwidth. It only does as a side effect of being lower latency than system memory. The TACC explanation of how queuing theory applies to CPUs makes it very obvious that memory bandwidth is inversely proportional to memory access times, which is why the cache has more memory bandwidth than system RAM. It is a side effect of the actual purpose, which is to reduce memory latency.

> 99% of the datacenter machines are not attached to the GPU. Does that mean that 99% of datacenter workloads are not "truly exceptional" for whatever the definition of that formulation and they are therefore mostly CPU bound?

99% is a made up number. Your question is nonsensical.

Not only you're ending up being very disrespectful but you're also pulling out the appeal to authority argument. Also something Brendan Gregg did on me here at HN meaning that no experience can substitute the amount of ego in some guys.

FWIW you can be let assured that you can't match my experience but that's not the argument I would ever pull off. I like to be proved wrong. This is a way I learn new things. BTW I designed my first CPU 15 years ago but during the career I learned to put my ego aside, discuss objectively, think critically and learn on my own reasoning mistakes from other people. Many of these points you are obviously lacking so this is a waste of time for me - I see no way to drive this discussion further but thanks anyway.

Intel's iGPUs were low end. Battlemage looks firmly mid-range at the moment with between 4060/4070 performance in a lot of cases.

None of this is magic. None of it is even particularly hard. There's no reason for any of it to get stuck. (Intel's problem was letting the beancounters delay EUV - no reason to expect there to be a similar mis-step from Nvidia.)

The intel problem was that their foundries couldn't improve the die size while the other foundries kept improving theirs. But technically nvidia can switch foundry if another one proves better than TSMC even though that doesn't seem likely (at least without a major breakthrough not capitalized by ASML).

https://www.techpowerup.com/gpu-specs/nvidia-gb202.g1072

Maybe there is a RTX 5090 Ti being held in reserve. They could potentially increase the compute on it by 13% and the memory bandwidth on it by 25% versus the 5090.

I wonder if anyone will try to solder 36Gbps GDDR7 chips onto a 5090 and then increase the memory clock manually.

Isn't not being kept a secret, its being openly discussed that they need to leverage AI for better gaming performance.

If you can use AI to go from 40fps to 120fps with near identical quality, then that's still an improvement

I've been using DLSS for FPS and racing games since I got my 3080 on launch and it works perfectly fine.

Frame gen might be a different story and Nvidia are releasing improvements, but DLSS isn't terrible at all.

So the biggest benefit is PCIe 5 and the faster/more memory (credit going to Micron).

This is one of the worst generational upgrades. They’re doing it to keep profits in the data center business.