The code was written back in 2012 and is using numbers from back then to _guess_ what future numbers would look like!
(I was fooled by it too)
See https://github.com/colin-scott/interactive_latencies/issues/...
"Seek + rotational delay halves every 10 years" just isn't true. We got rid of the >7200 drives and are basically sitting still on that metric. And hard drives weren't at 400MB/s in 2012 either. (The disk bandwidth section also cites a 2012 presentation with correct numbers, but something went wrong in translation.)
If anything, the takeaway is that things like memory/cache access, branch prediction failures, and mutexes have gotten more expensive. They didn't scale while the rest of the CPU sped up!
But even that isn't really true, because it doesn't tell you anything about branch prediction hit/miss rate, memory prefetching, instruction reordering, hyperthreading, etc. For a fair historic comparison you'd have to look at something like the cumulative time the execution units of a core are stalled due to waiting for something.
Honestly, the actual numbers aren't even all that relevant. It isn't really about the time individual operations take, but about the budget you have to avoid a latency penalty.
There are two ways to look at these sorts of numbers, "CPU performance" and "Systems performance". To give an example from my history;
NetApp was dealing with the Pentium P4 being slower than the Pentium 3 and looking at how that could be. All of the performance numbers said it should be faster. They had an excellent OS group that I was supporting who had top notch engineers and a really great performance analysis team as well, the results of their work was illuminating!
Doing a lot of storage (and database btw) code means "chasing pointers." That is where you get a pointer, and then follow it to get the structure it points to and then follow a pointer in that structure to still another structure in memory. That results in a lot of memory access.
The Pentium 4 had been "optimized for video streaming" (that was the thing Intel was highlighting about it and benchmarking it with.) in part because videos are sequential memory access and just integer computation when decoding. So good sequential performance and good integer performance gives you good results on benchmarking video playback.
The other thing they did was they changed the cache line size from 64 bytes to 128 bytes. The reason they did that is interesting too.
We like to think of things a computer does as "operations" and you say "this operation takes 0.x second, I can do 1/x operations per second." And that kind of works except for something I call "Channel semantics" (which may not be the official name for it but it's in queuing theory somewhere :-).
Channel semantics have two performance metrics, one is how much bandwidth (in bytes/second) a channel has, and the other is what is the maximum channel operation rate (COR) in terms of transactions per second. Most engineers before 2005 or so, ran into this with disk drives.
If you look at a serial ATA, aka SATA, drive it was connected to the computer with a "6 Gb" SATA interface. Serial channels encode both data and control bits into the stream so the actual bytes that go through a 6 gigabit line can be < 600 Mb when the encoding is 10 bits in the channel for every 8 bits sent (called 8b/10b encoding for 8 data bits per 10 channel bits (or bauds)). That means that the channel bandwidth of a SATA drive is 600MB per second. But do you get that? It depends.
The other thing about spinning rust is that the data is physically located around the disk, each concentric ring of data is a track, and moving from track to track (seeking) takes time. Further you have to tell the disk what track and sector you want, so you have to send it some context. So, if you take the "average" seek time, say 10mS, then the channel operation rate (COR) 1/.010 or 100 operations per second.
So let's say you're reading 512 byte (1/2K) sectors from random places on the disk, then you can read 100 of them per second, but wait 100? That would mean you are only transferring 50 kB per second from the disk, what happened to 600MB?
Well as it turns out your disk is slow when randomly accessed, it can be faster if you access everything sequentially because 1) the heads don't have to seek as often, and 2) the disk controller can make guesses about what you are going to ask for next. You can also increase the size of your reads (since you have extra bandwidth available) so if you read, say 4 kb sectors, then 100 x 4 kB is 400 kB/second. And 8 fold increase just by changing the sector size. Of course the reverse is also true, if you were reading 10 Mb per read, at a 100 operations per second that would be 1000 Mb per second which is 400 Mb more than your available bandwidth on the channel!
So when your channel request rate is faster than the COR and/or the data size requests are greater than the available bandwidth, you are "channel limited" and you won't get any more out of the disk no matter how much faster the source of requests improves its "performance" in terms of requests/second.
So back to our story.
Cache lines are read in whenever you attempt to access virtual memory that has not been mapped to the computer's cache. Some entry in the cache is "retired" (which means over written, or written out first if it has been modified, and then overwritten) and the new data is read in.
The memory architecture of the P4 has a 64 bit memory bus (in 72 data bits if you have ECC memory) That means every time you fetch a new cache line, the CPU's memory controller would to two memory requests.
Guess what? The memory bus on a modern CPU is a channel (they are even called "memory channels in most documentation") that are bound by channel semantics. And while Intel often publishes it's "memory bandwidth" number, it rarely would publish its channel operation limits.
The memory controller on the P4 was an improvement over the P3, but it didn't have double the operation rate of the P3. (it was like 20% faster as I recall, but don't quote me on that.) But the micro-architecture of the cache doubled the number of memory transactions for the same workload. This was especially painful on code that was pointer chasing because the next pointer in the chain shows up in the first 64 bytes and that means the second 64 bites the cache fetched for you are worthless, you'll never look at them.
As a result, on the same workload, the P3 system was faster than the P4 even though on a spec basis the P4's performance was higher than that of a P3.
After doing the analysis some very careful code rewriting and non-portable C code which packed more data in the structures into the 128 byte "chunks" where both 64 byte halves had useful data in them. Improved the performance enough for that release. It also was that analysis that gave me confidence that recommending Opteron (aka Sledgehammer) from AMD with its four memory controllers and thus 4x memory operations per second rate was going to vastly outperform anything Intel could offer. (spoiler alert: it did :-))
Bottom line, there are performance" numbers and there is system performance* which are related, but not as linearly as certainly Intel would like.
The P4 was a design that reached higher cycle rates, but at the cost of a lower IPC such that the net performance was a regression. In particular the pipeline depth was such that any dependency stalls or flushes eroded CPI severely.
There's a lot that went into this mistake. Having a high Ghz number for marketing purposes was part of it, but also in this era Intel Labs had demonstrators of 10 Ghz ALUs running, so there was some reason to think this architecture might prevail eventually. In any case, by the time it launched retail, Intel knew they had a lemon, but couldn't change course without building up a new microarchitecture.
Back in the early 2000's several of the P4 architects were quite active on comp.arch, and quite forthright about the mistakes they made. There were lots of interesting and insightful posts, but sadly I think you'd have to do some considerable digging in the deja archives or whatever it's called now to find the subthreads.
If you're interested in this sort of stuff, Hennessy and Patterson's Computer Architecture: A Quantitative Approach is the go to textbook.
For example, the top right label, "Read 1,000,000 bytes sequentially from SSD" shows as 31,000ns when I load the page, but when I move the slider to the left and back to 2020, it shows a different value, 49,000ns.
This happens for about half of the values for the data labels in chrome, firefox, and safari. The initially shown values are neven shown again once the slider is moved, even when it is moved back to the starting position.
var year = new Date().getFullYear();
Uh or storage and networking? Not sure why you would leave that out, since they're the bottleneck in many programs.
The slowest things are the first things you should optimize
I literally refuse to run a machine that boots its main OS from spinning disk anymore. The 60 bucks to throw an SSD into it is so incredibly cheap for what you get.
My wife's work gave her a (fairly basic but still fine) thinkpad - except they left the main drive as a fucking 5400rpm hdd. Then acted like assclowns when we repeatedly showed them that the machine is stalling on disk IO, while the rest of the system is doing diddly squat waiting around. I finally got tired of it and we "accidentally" spilled water on it, and somehow just the hdd stopped working (I left out the part where I'd removed it from the laptop first...). Then I just had her expense a new SSD and she no longer hates her work laptop.
Long story short - Storage speeds are incredible compared to what they were when I went to school (when 10k rpm was considered exorbitant)
It's currently been running for 23 days. Booting takes ~15 seconds even on spinning rust for a reasonable linux distro, so I'm not going to stress about those 15 seconds every couple of months.
total used free shared buff/cache available
Swap: 37Gi 617Mi 36Gi
5.1 gigabytes mostly just file cache. As a result, everything opens essentially instantly. For a bit better experience, I did a:
find ~/minecraft/world -type f -exec cat {} > /dev/null \;
to forcibly cache that, but that was all I did.
Not to mention the spinning rust is cheaper.
I can get a 60gb ssd for under 20 bucks (I just checked 18.99 on amazon - even I'm a little surprised at how cheap that is)
That 32gb of ram is at least a crisp hundred (which also - fucking mind blowing. First time I ever added a part to a PC, it was to add 256mb of ram, and that was huge deal)
Also I'd be concerned about any SSD you can get for that price.
And, frankly, you don't need that much RAM to have a pleasant desktop experience. Most of that is just for the gaming. Just a gig or two of file cache for the hot stuff. Which is what your OS usually does anyway.
The tower my workplace bought me is also spinning rust, and they for some reason put an enormous amount of RAM in it too. Only 5 gigs are actually being used for caching. The OS can't figure out what to do with the rest after having, I suppose loaded browser (few hundred megs), VMWare, OpenOffice etc. Even if I was running something more bloaty than Mate I doubt I could find a way to burn more than another gigabyte :)
Because, I'm also spoiled on RAM so I don't have to pick, but if I haaad to pick - I'd take a machine with 8gb of RAM and an SSD over 32gb of RAM and an HDD almost any day (actually - literally any day except when I'm running our full service stack at work, which needs about 12gb of RAM to be workable)
If I have a bunch of media - sure, throw it on a spinning 2tb drive or something. I'm not saying don't use cheap storage, I'm just saying... try the SSD. It costs about the same as coffee for the week.
Everything is linux too, but I find it hard to believe that would matter. I mean, NTFS is a notoriously slow file system (Hedgewars, Firefox and Minecraft all had to rearchitect to handle its horrendous handling of many small files), but the reasons for it I think would not change that much with SSD.
The thing is. How would you really notice a difference after first load of an app? SSD, spinning rust, linux, everything is mem caching these days... It's really hard to tell unless you're actually memory constrained.
It's quite likely that if I had to work on an enormous number of new files, I might notice something? But my software projects aren't like that - the games and videos are, but they wouldn't fit on an SSD anyway. That's why I have an 8 terabyte hard drive.
I absolutely notice a pretty major difference. If you don't - hey, if it's not broken, don't fix it I guess.
But, random access reads of many small files scattered across the disc, obviously SSD will do much better. Sequential reads (large files, non-fragmented filesystem) it's not so clear.
Maybe it's simply that Linux, as well as being better at file caching and not wasting RAM, is also better at not being fragmented. Perhaps I have my OS filesystems to thank for my indifference.
On that front I guess it's also saving me money by allowing me to enjoy ginormous cheap discs :)
And 2-3x faster is low. For my wife's machine, moving from a 5400rpm spinning disk to a relatively cheap samsung ssd made effective read times about 12x faster. Technically a 5400 rpm drive can do ~75mb/s, but that mostly ignores seek time, and assumes sequential reads. In reality, it was often doing less than 40mb/s. The SSD caps at about 540mb/s, but in reality it mostly does around 450.
It's the difference between having MS teams take ~5 seconds to load, vs more than a minute (locking most of the machine for that period as well).
I last launched Teams on this machine a couple of days ago.
For a "fair" comparison (keep in mind, again, the whole point I was making that RAM dominates these days), I then ran:
sync; echo 3 > /proc/sys/vm/drop_caches
And launched Teams again. It opened in between 8-10 seconds with ALL OS caching of disc dropped.
I think what you're seeing here is Windows being helped along by SSD making it usable...
It reminds me of the change we made in Hedgewars. We had no idea it was taking 20-30 seconds to launch our gui in Windows since it opened instantaneously in Linux even with all caches flushed. We had to add lazy page loading to get reasonable performance on Windows - it was probably all about NTFS being insanely slow (and maybe crap at laying out data reasonably).
*Edit - I updated my time estimate upwards slightly for teams after testing the flush several times and my own counting skills.
The flush was also a good test of launching firefox, since I did it once between to update nightly, it also opened very quickly post flush. Barely noticeable delay.
I remember the first memory stick I bought (a measly 4mb of RAM) and even wholesale via a friend who worked for a large PC repair shop, it was still $450 give or take.
Now 30 years later, my most recent memory purchase was eight 32gb ECC RDIMMs for about the same price. So not only did the amount of ram per dollar increase 64,000x, it’s also WAY faster and more resilient.
But I went a step further: the mother-in-law. She had a lower grade I3 laptop from around 2008 that took 20 minutes to boot Windows 10. It now boots in 15 seconds with a stock 512GB SSD.
I am being rewarded by both (but in very different ways).
I'm going to hell for this.
Lately my vendor discussions have centered around how much work you can get done with a machine that has half a terabyte of RAM, 96 cores, and 8 NVME SSDs (it's a lot). my college box: 40MB disk, 4MB RAM, 1 66MHz CPU.
Use cheap storage for when you want cheap performance (like those pesky dancing cats :) )
Can you explain how to square that with the link saying compressing data hasn't gotten any faster since 2005?
Compression may be bottlenecked by ram (L2 or L3 cache) access speed, or maybe even dram (main memory) access speed.
SSD was the biggest upgrade in performance in the past 15 years. There are still lots of system running on old HDD.
And the chart shows why I have been calling attention to latency computing for 10+ years. I hope the next stage is to reduce latency from Input, from keyboard to touch screen, and display input, and the actual display panel itself. On a 120hz screen even if you assume zero latency in everything else you still have a minimal of 8.3ms delay. Which is still quite obvious [1] once you put things in motion. And I think it is so hard there is enough room for at least another 10 years of hardware improvement.
[1] Applied Sciences Group: High Performance Touch
I think there are other effects too. Things like larger caches, smarter branch predictors, wider cores, etc. My 2021 processor (Apple M1 Pro) benches roughly twice as fast as my 2015 processor (Intel Core i5 2.7Ghz) in both synthetic and real-world single-core workloads.
That's not as big an improvement as we would have seen over that time period in the 90s, but it's a significant improvement none-the-less.
100%, we can no longer rely on faster processors to make our code faster, and must instead write code that can take advantage of the hardware's parallelism.
For those interested in learning more about Why Amdahl's Law is Important, my friend wrote an interesting article on this very topic - https://convey.earth/conversation?id=41
https://medium.com/@donhopkins/bill-joys-law-2-year-1984-mil...
It was never really true. PS It is formulated as 1984 not 1884, though. 1884 throws off MIPS calculations by 2^100 or 10^30.
idk, sure seems like we could have 1-2 cores (permanently pegged?) at 5 ghz for UI/UX then ($money / $costPerCores) number of cores for showing off/"performance" by now. But the OEMs haven't gone that way.
EVERYTHING that chip did, could in fact be done with a GPU today. It isn't, for the most part, because window systems evolved to be CPU driven, although a lot of phones these days do the UI in the GPU, not the CPU for this same reason. There is a fun program for HW engineers called "glscopeclient" which basically renders its UI via the GPU.
So I'm wondering if I misread what you said and are advocating for a different GPU micro architecture or perhaps an integrated more general architecture on the chip that could also do UI like APUs?
Not sure the OEM's have what it takes to do that without a serious rethink, and that is what an M1 basically is.
And say they do...
Amdhal's law is still going to be a thing because people are unlikely to believe having two cores running balls out whether essentially idling or pushing high priority interaction... they will see it very poor energy management, and would want tasks moved onto and off those always fast cores at a minimum to reduce what they see as waste.
(well, speed of signal on a PCB track is roughly 2/3 light and determined by the transmission line geometry and the dielectric constant, but you all knew that)
But I dont think hollow cable are anywhere close to commercial launch.
1) The 'ns' next to the box is a graph legend not a data label (normally that would be in a box labeled legend to distinguish it from graph data)
2) The weird box and rectangle thing on the top is a slider, I didn't notice that until I was looking at the code and said "what slider?"
3) The only changes from 2005 to present are storage and networking speeds.
What item #3 tells you is that any performance gains in the last decade and a half you've experienced have been driven by multi-core, not faster processors. And that means Amdahl's Law is more important than Moore's Law these days.