News for hackers. Stuff that matters.

A comment on An Intro to DeepSeek's Distributed File System. Parent comment

Re:An Intro to DeepSeek's Distributed File System by __turbobrew__ on Thursday April 17, 2025 @02:11PM (#43720342)
With latest ceph releases I am able to saturate modern NVME devices with 2 OSD/NVME. It is kind of a hack to have multiple OSD per NVME, but it works.

I do agree that nvme-of is the next hurdle for ceph performance.

[ Parent | on HN ]
Re:An Intro to DeepSeek's Distributed File System by sgarland on Friday April 18, 2025 @09:48AM (#43728082)
I thought the current recommendation was to not have multiple OSDs per NVMe? Tbf I haven’t looked in a while.

I have 3x Samsung NVMe (something enterprise w/ PLP; I forget the model number) across 3 nodes, linked with an Infiniband mesh network. IIRC when I benchmarked it, I could get somewhere around 2000 MBps, bottlenecked by single-core CPU performance. Fast enough for homelab needs.

[ Parent | on HN ]
Re:An Intro to DeepSeek's Distributed File System by __turbobrew__ on Friday April 18, 2025 @11:58AM (#43729277)
https://ceph.io/en/news/blog/2023/reef-osds-per-nvme/

I benchmarked 1 vs 2 OSD and found 2 OSD was better. I don’t think it is recommended to run more than 2 OSD per NVME.

[ Parent | on HN ]