IOPS is measuring throughput for parallel operations.

Optane SSDs have extremely low latency, which is critical for sequential operations.

Most operating systems have long phases of largely single-threaded synchronous I/O, or nearly so. They certainly can't maintain queue depth required to even approach 500K IOPS.

At queue depth 1 the 980 pro delivers 20k reads per second / 45k writes per second. That still sounds like a huge amount. Optane is faster, but is it enough to matter outside some extremely demanding niches?

The other reason Optane excels is latency. This is why single-threaded use cases, like system boot, love it.

IOPS do not tell the whole story here. Consider disk A, at 1M IOPS, against disk B, at 1k IOPS. Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. Which disk is higher performing?

You might say they're both capable of the task, so just get the cheaper one. This is wrong. Assume, for sake of a simple argument, that disk A can respond in 1us to these requests, and disk B in 1ms. That is, they'd be able to hit "max IOPS" on these requests, if only you have enough. Then disk A services the whole thing in 500us per second, or 0.05% used, and disk B in 500ms per second, or 50% used.

So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care.

Either point of view can be correct, at different times or for different systems. Ignorance that both points of view exist is how things like bufferbloat are everywhere now.

> The other reason Optane excels is latency.

It's not an "other reason". IO operations at queue depth 1 are a direct measurement of latency. Queue depth 1 is not about throughput at all.

> Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly.

I will assume the workload is light in CPU too, since spending half the time waiting for the disk is apparently not an instant disqualification.

> So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care.

If you're waiting on a bunch of results and disk A gets them to you significantly faster, it sounds like your initial premise of "only 500 IOPS" was wrong or misleading.

If you have a response time you want to meet, then you should measure IOPS with a sliding window of that many milliseconds. If it's still 500 at most, then disk B will still be fine for meeting your target.

If it turns out that "500" actually comes in bursts of 50 operations, and you want to be done in 10 milliseconds, then you should label your workload as "5k IOPS". Put in a drive that does at least 10k and you'll see all calculations done in under a frame; basically perfect latency. A drive a million times faster would have no practical difference.

IOPS are actually meaningless assuming conventional filesystem APIs. In the context of those APIs, IOPS are just a different way of writing MB/s except you get to show off that your block sizes are smaller.

Latency is a much bigger factor with conventional filesystem APIs. If you had fully async filesystem APIs that read offset x1 on file f1, offset x1 on file2, offset x100 on file f100 all at the same time then you would see linear scaling and the difference between Optane and NAND would grow smaller. However, that would require learning a new and much harder paradigm (you now have to be aware of parallelism inside SSDs). Optane is a simple upgrade that requires no software changes.