Again, terms and conditions.
The slow part for memory access isn't how much memory can be piped over the wire but rather how fast a request for the next chunk of memory can be issued. So while it's possible to get those 480 multiplications that relies on the cache being warm and the memory access being predictable to the CPU. If either of those two constraints are violated, then the number of multiplications is severely decreased.
In other words, the more frequently you have to tell the memory controller to get more memory, the less work you can do.
To put it in perspective, it costs around 100 cycles to load something from main memory. Imagine you are talking about something like an n body simulation that may be accessing memory all over the heap. In the worst case, you are looking at 4 multiplications (if multiplying by a scaler) and perhaps even just 2 multiplications if you need 2 memory pulls per multiplication.
Caches don't just help, they are necessary to come anywhere near achieving that 480 number.
Just a little history lesson.
Once upon a time, the memory controller didn't exist on the CPU, it was on a separate chip (the north bridge). Whenever the CPU wanted to load memory, it'd send a request to the north bridge and the north bridge would send that request to main memory.
One of the huge improvements to CPU performance for both Intel and AMD was integrating the memory controller onto the CPU. Why? Because they shortened the distance a request for memory, and the memory itself, had to travel significantly. Electrical signals travel at about 2/3 the speed of light and if your memory is 0.1 meter from the CPU, you can see how many requests for a new chunk of memory can really slow down everything.
This is fundamentally why nVidia's big GPGPU processors they sell to datacenters come chock full of memory. Because they want to limit the amount of time and times the GPGPU has to reach back into main memory to process stuff. It's far quicker to load up the 20gb of ram on the card in one go then to need to constantly pull that data over the PCIe line.
To use an analogy: I'm not asking how far a car can go on the highway if accelerating from zero, but how far it can move if moving at highway speeds.
Nonetheless, the latency to main memory is also a great example of how bad programmers' intuitions are! A processor can perform thousands of operations in the time it takes for it to wait for just one memory access.
RAM is the new disk, and disk is the new tape.
PS: There was an article on here just a couple of days ago about how "handles are better than references". I didn't see many comments pointing out that this doubles the number of memory requests required when chasing pointers, halving the overall performance!
> To use an analogy: I'm not asking how far a car can go on the highway if accelerating from zero, but how far it can move if moving at highway speeds.
The terms and conditions keep applying. The everyday real world conditions implied by "highway speed" include regular cache misses and branches.
It's not about "start" or "acceleration". This massively impacts the steady state speed.
If you want a car analogy, tight vector packing compares to normal code about as well as looking at the speed and width of the flame front across a cylinder and using that to measure fuel consumption.
It sounds more like testing whether you know the speed of DDR5 without looking it up, and doing a bunch of basic unit conversions.
And your analysis doesn't even cover latency (which GP mentioned), which apparently according to my first google result is ~90ns. So unless your data is already in a cache it might not even be fetched within the requisite time.
Your question sounds like a trick question because there are so many nuances and gotchas, and if you're asked this question without a concrete context and the problem one's trying to solve, it's basically a check of whether the interviewee's mental model of computation matches the interviewer's. And as this thread shows, even among (presumably) competent people, the factors to consider can differ quite a bit.
What I’ve seen, intuition wise, on where slowness is in computation it’s usually along the terms of “how many FLOPs can this do?” or “How much bandwidth does this have?”. But that’s a tiny fraction of the story. Very little computation is boring math operations.
It’s true, CPUs are absurdly fast at multiplying 2 numbers together. That’s not what trips them up. What trips them up is getting those two numbers and determining when those 2 numbers should be multiplied. Those two problems are so hard that CPUs are often guessing, running the calculation, and then backtracking if they guessed wrong.
And that action has lead to side channel attacks (spectre, meltdown) which can leak information about data to malicious software.
This is why such a question doesn’t have either an intuitive or non-intuitive answer. It’s very strictly “it depends”. And if you boiled it all the way down to “no, what’s the absolute max it can do” you are now talking about software problems that don’t exist/aren’t valuable.
Consider, what execute the add instruction faster, a 3.4 ghz CPU from 2004 or a 3.4ghz CPU from 2023? The answer is they are the same. Both do the add in 1 cycle. Did you learn anything about the performance of a 2004 CPU vs a 2023 CPU? Heck no. That’s because how fast the CPU can execute instructions has almost nothing to do with how fast it can process data. (Yes, I’m aware that even the above statement is somewhat incorrect as x86 CPUs do weird things with adds where they’ll run multiple additions per cycle with some clever reordering… Not even that example can be simple. Assume they didn’t).
The trivia question is a boring question. The actual interesting discussion is “so WHY aren’t we doing multiplications that fast?”. That is what can actually lead to deeper understandings about what slows things down in computing. There’s a reason the 10ghz CPU never surfaced. It’s not because we couldn’t make it, it’s because we couldn’t keep it busy.
That’s 64 bytes per nanosecond, or 16 floats per ns, and hence 480 in the time needed for light to go 10 meters.
That’s the “base” performance. Caches and registers would improve on this significantly in realistic scenarios.
Again, this is why I like this thought experiment! It’s not a trick question. It’s testing intuitions.
People just think it’s a trick because their intuitions are so ridiculously out of whack with reality that they’re looking for the sleight of hand instead of reexamining their own notions.