News for hackers. Stuff that matters.
Re:Torus development has been stopped at CoreOS by jacques_chester on Sunday February 19, 2017 @05:05PM (#13682320)
At a glance, Torus was intended to be a distributed file system. As I understand it, distributed file systems are easier than distributed block systems, but harder than distributed blob systems.

A blob system is all-or-nothing. You create or replace the entire blob at once. This makes bookkeeping and replication much easier for the implementer.

A filesystem supports much richer semantics, including the ability to seek parts of files and modify small regions of files. You need a lot more mechanics to maintain consistency across a network.

A block store is difficult because you're trying to work at very high speed on very small units of state wooshing back and forth willy-nilly. You don't get to rely on any of the higher semantics provided by a filesystem or blobstore, since you're pretending to be a magical harddrive.

I am often wrong in these matters, as an interested outsider, so I'd be happy to receive correction.

Re:Torus development has been stopped at CoreOS by umamukkara on Monday February 20, 2017 @08:01AM (#13686844)
Disclosure: I work for OpenEBS project

Torus was intending to write distributed block storge that is container native. Metadata management using key value (KV / etcd) method is increases the complexity and not new. Ceph tried it.

OpenEBS uses a novel approch, linux sparse files for managing the blocks of a volume. Fork of Rancher longhorn. The issue of managing the large scale distributed block storage metadata is solved easily throught he management of the files (not blocks).

https://blog.openebs.io/torus-from-coreos-steps-aside-as-clo...

Re:Torus development has been stopped at CoreOS by fh973 on Monday February 20, 2017 @12:23PM (#13689029)
What I don't get around the new efforts around container-native storage: if you decide to build a new container-native storage system, why would you aim for block storage and not file storage?

Block storage is not exactly a great fit for containers as you can't access its file systems from multiple hosts and fail-over is a hassle (forced remount, fsck).

Re:Torus development has been stopped at CoreOS by umamukkara on Tuesday February 21, 2017 @04:26AM (#13694395)
Container native storage has two aspects. One, the storage that container uses, (Like Docker uses Device Mapper, Overlay2 etc.) and two, the persistent storage that the applications inside containers need.

Both of them need to be truly container native. Portworx is attempting LCFS to provide a container native storage for containers itself https://github.com/portworx/lcfs. So, you are right. You would need a container-native storage (file system for Docker) for running containers.

OpenEBS is targeting providing containerized storage (persistent+block storage) for applications in containers. OpenEBS builds a storage volume as a container and presents the volume-container as part of the K8s POD. This way the the storage persistence problems are resolved by K8S orchestration ingelligence written for application PODs.

Ofcouse, OpenEBS containers will use LCFS when it is ready.

Re:Torus development has been stopped at CoreOS by wmf on Tuesday February 21, 2017 @03:33PM (#13699621)
File systems are much harder than block storage due to the large amount of metadata and the consistency/durability requirements for metadata. So a lot of systems just build block storage and format ext4 on top or put NFS+ZFS on top (but that's not very distributed).
Re:Torus development has been stopped at CoreOS by fh973 on Monday February 20, 2017 @12:16PM (#13688960)
Working on a distributed file system (Quobyte), I'd say the order of complexity is file > block > object. Why? Because a drop-in replacement for a local file system also needs to be good at high-performance concurrent block IO, otherwise you wouldn't be able to run applications like databases.
Re:Torus development has been stopped at CoreOS by jacques_chester on Monday February 20, 2017 @06:46PM (#13692049)
So am I wrong in thinking FS semantics can give some hints on how to optimise activity?

That is: a pure block store must treat every operation with identical care, as it has no insight into the higher-level meaning. But a filesystem can, for example, give consideration to whether it's a write, read, create or appending op; where in the file it's happening; the order of operations; metadata vs file contents and so on.

Again, I am only distantly familiar with any of this.

Re:Torus development has been stopped at CoreOS by zzzcpan on Monday February 20, 2017 @07:01PM (#13692109)
"block store must treat every operation with identical care"

Yes, and because of that it's easier to do a block storage, than a filesystem. But sure, filesystems have enough semantics to potentially scale across multiple data centers and probably be performant enough for many apps to tolerate those latencies. However, the level of complexity would be similar to that of Google's Spanner.

Re:Torus development has been stopped at CoreOS by jacques_chester on Friday February 24, 2017 @01:06PM (#13725983)
Thankyou.