News for hackers. Stuff that matters.
Re:The NixOS Foundation’s Call to Action: S3 Costs Require C... by antongribok on Saturday June 03, 2023 @10:11AM (#36176818)
While I totally agree you can easily handle this with Ceph, I would never do it with 2x replication on spinning rust. You can maybe do 2x replication with SSDs if you really know what you're doing and you have data replicated on multiple Ceph clusters.

Also, your math does not quite add up if you want to store 425TiB. You'd be filling up the Ceph cluster to 95% which would be impossible (you'll start losing OSDs way before that).

If this was me, I would definitely do this on Ceph, but I would opt for a more traditional erasure coding setup of 8+3. For that, you'd need 12x SX134s as a minimum starting point.

I suppose you could start with an EC profile of 2+2 or 4+2...

I do have a serious question on Hetzner:

Do they offer any private networking options? That page only lists a single 1 Gbit connection. I don't think that would work well with Ceph. It's doable, sure, but I would want to support such a cluster, not with 160TB raw per box.

Re:The NixOS Foundation’s Call to Action: S3 Costs Require C... by justinclift on Saturday June 03, 2023 @11:29AM (#36177433)
> Do they offer any private networking options?

Doesn't seem like it for their dedicated servers, not 100% sure though.

Their docs indicate that upgrading the 1Gbit ethernet link to 10GBe is an available option:

https://docs.hetzner.com/robot/dedicated-server/network/10g-...

Re:The NixOS Foundation’s Call to Action: S3 Costs Require C... by chpatrick on Saturday June 03, 2023 @12:45PM (#36178174)
You can upgrade them to 10gb NICs.
Re:The NixOS Foundation’s Call to Action: S3 Costs Require C... by CoolCold on Sunday June 04, 2023 @05:44AM (#36184321)
Internal networking in simple case provided via VLANs on that 1gbit interface via their vSwitch.

Extending with additional NICs should be possible from what I've read but not used myself.