What are your speeds? Do you rub ceph FS too?
I'm trying to do similar.
Re:Ceph: A Journey to 1 TiB/s
by antongribok on Saturday January 20, 2024 @12:56AM
(#39065097)
It's been a while since I've done some benchmarks, but it can definitely do 40MB/s sustained writes, which is very good given the single 1GbE links on each node, and 5TB SMR drives.
Latency is hilariously terrible though. It's funny to open a text file over the network in vi, paste a long blob of text and watch it sync that line by line over the network.
If by "rub" you mean scrub, then yes, although I increased the scrub intervals. There's no need to scrub everything every week.
Rub, scrub... Was really supposed to be run but it got the answer I was looking for!
Right now, I'm doing about half of that but I am rcloning data between 2 ceph clusters. (Proxmox and a arm rook ceph cluster).
My household is already 100% on Linux, so having a native network filesystem that I can just mount from any laptop is very handy.
Works great over Tailscale too, so I don't even have to be at home.
[1] I run a large install of Ceph at work, so "easy" might be a bit relative.