pierrebarre

Or just use https://github.com/Barre/ZeroFS

Works pretty awesomely with git workloads, even Linux sized trees.

> i also dont really get the point of zerofs. Seems like you are building a posix fs on top of s3 that is built on top of a posix fs

That can literally be the stack, but the same argument could be made about object storage itself: it ultimately writes data to local disks, so why not write files there directly?

The value of object storage isn’t just its API. It’s the durability, distribution, failure handling, and capacity behind that API. ZeroFS lets POSIX applications use those properties. Its server and cache are replaceable and the bucket remains the source of truth.

If one machine and its disks are enough for your data and reliability needs, then yes, a local filesystem is simpler. ZeroFS isn’t intended to improve that case.

Yes, I think it’s worth benchmarking.

zerofs mount has durable fsync and fcntl byte-range locking.

Inotify works for changes made through the local mount. It won’t report changes made through another client, though.

Small writes operate on 32 KiB extents. Partial overwrites are read-modify-write, but the FUSE writeback cache helps combine them, and the resulting extents are packed into segments rather than producing one S3 PUT per write.

I’d be interested in the results if you try it.

Hardlinks work as you’d expect: multiple paths point to the same inode and data, so a write through one path is visible through the others. Two separate files with the same contents are stored separately.

There’s no deduplication, either whole-file or block-level. That’s intentional, mostly because of the impact it would have on locality.

If by CoW you mean reflinks, those aren’t currently planned either. They avoid the content matching part of deduplication, but still require sharing extents between files and come with similar locality and complexity tradeoffs. Internally ZeroFS is copy-on-write, with immutable segments and checkpoints, but that isn’t exposed as reflinks.

Thanks!

stat doesn’t pull the file contents from S3; it only accesses the metadata tree, which is usually cached.

I haven’t benchmarked Borg or Restic specifically. Sequential writes can comfortably reach several Gbit/s. For follow-up runs, if they only stat unchanged files, that should stay entirely in metadata.

The default Redis/Valkey configuration should work fine for conditional_put. NFSv4 is unlikely for now. It would add a lot of surface area, and I’m pretty happy with where the 9P extensions are today.

That’s a fair concern. The closest thing right now is a deterministic simulation suite that injects storage faults and crashes at arbitrary points, then checks the recovered data against reference models. It runs hourly with fresh seeds.

CI also runs pjdfstest, xfstests, stress-ng, ZFS scrubs, and Jepsen crash/failover tests: https://github.com/Barre/ZeroFS#testing

ZeroFS is still pretty young as storage projects go, so I completely understand wanting to see it prove itself over time.

Author here. Thanks for posting this!

It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.

Hi,

The mechanism is explained here: https://www.merklemap.com/documentation/how-it-works

I am working on https://www.merklemap.com/ A subdomain / CT search engine.

Shameless plug: https://github.com/Barre/privaxy

It’s a proxy that does just this.

Shameless plug: https://github.com/Barre/privaxy

It can be easily configured to block any host or path.

What are the things that you think are best handled by the browser while negotiating a session?

Privaxy is matching chrome on https://badssl.com/dashboard/ and https://www.ssllabs.com/ssltest/viewMyClient.html except that Privaxy doesn't support older TLS versions and poorer ciphers.

It does not mean that there is not a single bug, but I do not think it is fair to completely discount this approach. Especially when the alternative is browser extensions which bring their fair share of trouble regarding trust, performance, limited capabilities or even security.