News for hackers. Stuff that matters.
Re:Ceph: A Journey to 1 TiB/s by magicalhippo on Saturday January 20, 2024 @01:03AM (#39065134)
They're benchmarking random IO though, and the disks can "only" do a bit over 1000k random 4k read IOPS, which translates to about 5 GiB/s. With 320 OSDs thats around 1.6 TiB/s.

At least thats the number I could find. Not exactly tons of reviews on these enterprise NVMe disks...

Still, that seems like a good match to the NICs. At this scale most workloads will likely appear as random IO at the storage layer anyway.

Re:Ceph: A Journey to 1 TiB/s by mrb on Saturday January 20, 2024 @03:25AM (#39065683)
The benchmark were they accomplish 1025 GiB/s is for sequential reads. For random reads they do 25.5M iops or ~100 GiB/s. See last table, column "630 OSDs (3x)".
Re:Ceph: A Journey to 1 TiB/s by magicalhippo on Saturday January 20, 2024 @05:05AM (#39066091)
Oh wow how did I miss that table, cheers.