I tend to think your number for PS is likely off.
Also, I am not sure how it will stack up against some cloud instance with bunch of ssd under software raid.
That is absolutely not representative of a real workload of mixed reads and writes, different block sizes, potentially different queue depths, all coming in on hundreds or maybe thousands of different volumes.
A single consumer Samsung SSD would hilariously crumble under a real workload, it will NOT deliver hundreds of thousands of IOPs in that environment.
Your benchmark screenshot is the equivalent of showing your pickup truck can do burnouts in the parking lot, and extrapolating that to think it could keep up with a Ferrari on the Nurburgring.
if you see bottleneck there, it could be that actual SSD speed is maybe irrelevant in your case since upstream software is not optimized.
Also 1GB test file is often fits in the SSD's RAM cache. Get iometer, 50R/50W%, blocks from 512 to 16k, at least half the size of the storage. Then you would see the real performance.
NB if you have random read way below the random write means you are measuring anything but the storage performance.
Sorry, lazy to learn how to use iometer, but you probably have SSD too and can report your results.
Okay, not a problem: https://imgur.com/a/teoPGrz
Real world performance[+Mix], Read&Write[+Mix]
One is Fujitsu DX200 S4 SSD SAN, other is HFM256GDJTNG-8310A.
Can you guess which is where?
>> but you probably have SSD too and can report your results.
Uh-uh!
I intended to show you the difference between a single NVMe drive and a SSD SAN (a bit old, but still very performant to handle ~900 VMs).
Sure, I can just ramp up queue depth and see some magical numbers, but for me the real performance is in everyday tasks and running CrystalMark isn't an everyday occurrence.
Did you guess which one is SAN?
I am bit confused how random 4k access is relevant to your task, it should be more like 1MB seq access likely.
My everyday task is tuning heavy data processing pipeline, and I am trying hard to achieve those q16t16.
Sorry? 900 VMs equals 100% full random access. There is no sequential access there, just as I said in my first comment.
> It is obvious there is a difference
Because of your comment[0].
This comment[1] pretty much summarized what I said in a more eloquent way.
it depends on workload, if they do most of the work in RAM, and most of fs traffic is snapshoting and restoring from snapshots, then you will get 99% io seq traffic. If they do some non-trivial fs operations, then you will get q16t16 io traffic. It is very unlikely you will get q1t1 random.
> Because of your comment[0]. > This comment[1] pretty much summarized what I said in a more eloquent way.
In my view you are jumping from topic (single ssd vs nas) to another topic (your speculations about benchmark not representing real world scenarios) and then back.
Running Linux LVM software RAID over 3 x Samsung NVMe SSDs. That's not a read-write measurement, but it's a satisfying number for a not particularly high end server.
(I use it for a side project's database engine experiments. That level of IOPS supports a very high random query rate.)
Doing so is a lot more complex and intensive than what a single consumer grade SSD can handle.
My point was just that if you wanted to get dedicated IOPs on AWS to match what you get with modern SANs, it’ll cost you far more.