> PCIe Gen4 does matter. M.2 NVMe has been read limited for a long time already (NAND bandwidth scales trivially). The I/O section of this article is basically nonsense.

I think the author's point was that storage is already plenty fast enough for many tasks. Personally, I cant feel the difference in storage performance between my NVMe system and older SATA SSD ones, despite NVMe being much faster.

> I cant feel the difference in storage performance between my NVMe

That's because for the most common tasks, real performance had not improved.

Manufacturers always display the write/read speed using many cores and with huge queue depth. The typical use case for that is copying files. It's around 3000MB/s

When starting OS and loading program you have tasks that use low queue and thread count, which is about 60-70 MB/s.

Genuine questions. I just don't know this area...

> Manufacturers always display the write/read speed using many cores...

Why does #cores matter? Once the IO is initiated it's handed off to some DMA/coprocessor stuff so the core can get back to starting other IOs without contention. Being clueless I can't see why a single core couldn't saturate IO bandwidth by just blasting out IO requests.

> ...and with huge queue depth.

Really stupid question now, I thought disk queues were queued up requests caused by issuing a buttload of IO requests, so why does a disk queue of say 20 actually make things any faster over a disk queue of just 2?

I know (correction, believe) with spinny disks a longer queue could be used by the controller to optimise access on the disk surface by proximity, but with SSDs that performance characteristic doesn't exist (access is uniform anywhere, I think) so that optimisation doesn't apply.

> Why does #cores matter?

If you're benchmarking 4kB IOs, then the system call and interrupt handling overhead means you can't keep a high-end NVMe SSD 100% busy with only a single CPU core issuing requests one at a time. The time it takes to move 4kB across a PCIe link is absolutely trivial compared to post-Spectre/Meltdown context switch times. A program performing random IO to a mmapped file will never stress the SSD as much as a program that submits batches of several IO requests using a low-overhead asynchronous IO API.

> so why does a disk queue of say 20 actually make things any faster over a disk queue of just 2?

Because unlike almost all hard drives, SSDs can actually work on more than one outstanding request at a time. Consumer SSDs use controllers with four or eight independent NAND flash memory channels. Enterprise SSD controllers are 8-32 channels. And there's some amount of parallelism available between NAND flash dies attached to the same channel, and between planes on an individual die. Also, for writes, SSDs will commonly buffer commands to be combined and issued with fewer actual NAND program operations.

Thanks, very informative!

One more thing, I'm surprised that 4KB blocks are relevant, I'd have thought that disk requests in benchmarking (edit: cos manufacturers like to cheat), and a lot of reads in the real world, would operate at much larger requests than 4K.

Is it that IOs are broken down to 4K blocks at the disc controller level, or is that done deliberately in benchmarking to stress the IO subsystem?

SSDs are traditionally marketed with sequential IO performance expressed in MB/s or GB/s, and random IO performance expressed in 4kB IOs per second (IOPS). Using larger block sizes for random IO will increase throughput in terms of GB/s but will almost always yield a lower IOPS number. Block sizes smaller than 4kB usually don't give any improvement to IOPS because the SSD's Flash Translation Layer is usually built around managing data with 4kB granularity.

> expressed in 4kB IOs per second (IOPS). Using larger block sizes for random IO will increase throughput in terms of GB/s but will almost always yield a lower IOPS number

and higher IOPs numbers give the marketing department the warm and fuzzies. Got it.

It is a good metric because it is a measure of random access. Bigger requests are a mixed measure of sequential performance and random access (and you can basically infer the performance for any io size from the huge request bandwidth and the smallest reasonable IO IOPS)

> Personally, I cant feel the difference in storage performance between my NVMe system and older SATA SSD ones, despite NVMe being much faster.

For the tasks I do, this is night and day, and lack of NVMe is reason enough to warrant buying new hardware.

We are crafting a new Server cluster and it looks like the Gen4 PCIe will give us a 40% performance boost for what is basically NVMe over Fabric using Mellanox 100Gb cards (Azure HCI aka S2D). Next year we will be building out another cluster and will hopefully use Mellanox 200Gb cards which is only possible due to Gen4.

Performance tests here https://www.servethehome.com/amd-epyc-7002-series-rome-deliv...

Ah, but that’s making assumptions about the user. There are lots of power users for whom IO bandwidth matters; high-end video editing for example.

And low-cores are not?! We have been core starved for almost a decade!

IO matters but peak bandwidth sequential reads are not such a limiting factor even for power users.

They - and any other subset - can get the PCIe 4 SSDs of course.

There are "lots" of other subsets where it stopped mattering a while ago though.

> PCIe Gen4 does matter. M.2 NVMe has been read limited for a long time already (NAND bandwidth scales trivially). The I/O section of this article is basically nonsense.

It probably does for a great many special use cases on servers or other specialty workloads. But for programming/gaming/normal productivity workloads it really doesn't. The net difference in load times/compile times/boot times is tiny even going from a cheap ssd to the latest pci4 ssds.

If you do huge sequential reads/writes, then yeah, large differences are getting unlocked.

Has anyone tried adding branch prediction to disk operations? While it would confuse a network, or other peripheral a preemptive random read from a disk ought to have minor impact.

Operating systems have supported readahead for a long time. It's even more benefit for spinning rust than SSDs because the benefit of a correct prediction is larger when the drive is slower. But it's also a trade off, because if you predict wrong you're wasting scarce I/O resources fetching useless data.

I wonder how much of Intel’s stagnation is due to their bringing CPU engineering back to the US from Israel. In other words putting the geniuses behind Netburst and Itanic in charge.

Can you elaborate? Do you know why the repatriation was decided? Can you explain why Netburst and (I’m guessing this is a pun on) Itanium were failures? Do you know what Israeli teams did better?

Netburst was the Pentium 4 architecture that introduce large execution pipelines that permitted high clock speeds thinking they could out run inefficiency. Latency, power and temperature however became mitigating factors.

Alongside Netburst development, Intel had the Israeli team develop a mobile first platform. The team took parts of the bus for Netburst and shit canned the rest if the design in favor of the Pentium III which they optimized the shit out of. They introduced concepts like microop fusion and developed the processor with the goal of sleeping whenever possible to conserve power.

The net result was the Banias platform which was incredibly efficient because it was incredibly fast. The original Core line was basically Banias with power saving turned off.

From memory, the Netburst team ignored power efficiency until power density concerns hard-capped their frequency targets (and this was on 180/130/90 nm!).

Itanium is often lazily pointed to as a failure by Intel, but in reality it was a hard task with decent execution.

Intel attempted to build an ecosystem around an entirely new architecture. Emphasis on smart-compiler extracted instruction level parallelism, supported by wide but less superscalar hardware. In 2001.

As an analogy, this is kind of like asking every driver to toggle a switch to manage the engine mapping while they drive. It could be more effective, but it's also a ton of work and relies on expertise being developed.

AMD delivered what the market really wanted in x64: something that looked exactly like x86, with a few modest improvements, and some 32-bit scaling limitations removed. But no big changes.

The Israeli teams are usually cited as the caretakers of the P6 (Pentium Pro) legacy, which evolved into the Pentium M, and eventually the Core architectures.

They chose to optimize for power efficiency, and so didn't hit the same power caps the Netburst team did.

Generally, the internet comments on this are armchair generaling with the benefit of hindsight. There are many reasons things could have gone differently.

Sometimes two teams work equally hard, but one chooses a path that leads to success and the other one that leads to failure.

Well they've now put the genius behind Zen in charge (Jim Keller), have they not?

>Today, the software _is_ there (well, mostly) and we can all take advantage of more cores.

Disagree. Not even "mostly". If I had a dime every time I saw some software stuck at 100% of one core... well, I'd have a few dimes a day.

Most consumers use their devices for media consumption, games, and surfing the web. ALL of these are heavily multi-threaded. Yes, even dear old Javascript is parsed and compiled in multiple threads, concurrently GC'd and uses web workers, service workers, and webassembly.

In the business world, all the commonly-used software I can think of is heavily threaded as well. Generally, the remaining software that comes to mind is simple stuff that could run on a decade-old machine without too much trouble.

What typical programs are you running that require so much single-threaded performance?

Using Acrobat DC, searching for content in all PDFs in a folder takes ages. It is both slow AND single-threaded, the disk is barely moving.

Tried with Foxit and a few others, same thing. I had to seek indexed search apps, and those are few and far between and generally poor quality - kudos to Recoll[1] as the standout so far. But I'd much rather just search within the reader.

Anyway, it's infuriating, this should go about x times as fast as there are cores.

[1]: https://www.lesbonscomptes.com/recoll/

Windows Update, Windows Defender Antivirus, NTFS compression, maybe more I've forgotten.

Also old (and not-so-old) games.

Well, that's not at all a fair comparison. Who knows what that one core is doing? Not all algorithms are parallelizable, in fact most aren't. So doing standard, everyday things in code might still max out a core.

Exactly. So single-core performance wins.

> - PCIe Gen4 does matter

Strong disagree there, for threadripper and epyc products based on zen2. There's use cases for two and four port 100GbE PCI-Express NICs in 1RU and 2RU sized x86-64 platform systems where you definitely want as much pci-express bus bandwidth as possible.

In addition to the obvious use cases like people putting multiple nvme pci-express bus ssds in pci-express slots. At the low end of the price market things like a pci-express 3.0 x8 card that holds four m.2 2280 SSDs in a $18 part.

First generation single socket epyc was already a huge improvement in number of pci-express lanes per CPU socket compared to equivalently priced intel product, if you were, for example looking at building a 2RU system with eight or twelve 100GbE interfaces to run juniper JunOS as VMX.

In the single socket higher-end workstation CPU market ($400 to $800 CPUs) Intel has for a very long time had a constrained number of pci-express lanes available per motherboard. In a setup with a single high end video card like a 1080ti or 2080 and a single nvme SSD, it doesn't leave a whole lot of I/O lanes left over.

I cannot follow your argument, so let me give examples: https://www.anandtech.com/show/14729/samsung-preps-pm1733-pc...

Samsung introduced a 8 GB/s SSD specifically for PCIe Gen4. I used to work in the SSD business many years ago and we could _trivially_ saturate PCIe Gen3 x4 but it wasn't economical to go to more lanes. (Bandwidth is easy, latency is hard).

Somehow I mistakenly read the parent comment as pcie 4.0 doesn't matter, and typed a whole bunch of paragraphs in answer to that. Point was that 4.0 and more lanes are a much needed improvement so I actually am in agreement with the parent comment. Oops.

Maybe I'm missing something, but PCIe Gen4 does seem like a big deal since it's effectively doubling the data rate of the previous gen. Meaning, once all/most of your peripherals are PCIe 4 compatible, you'll theoretically have double the PCIe lanes, on account of only needing half as many.

One immediate benefit that comes to mind, is storage controllers. With 8x spinning disks, you might get away with using a single lane which can provide almost 2 GByte/s bandwidth. And controllers previously using 8xGen3 lanes could switch to 4xGen4 without any loss in performance.