jjobi

SSD was used as the backend device for the tests. If a file is used for LCFS backend, the performance will be limited by the underlying file system in which the file is hosted. LCFS is capable of working directly on top of device and can queue I/Os efficiently and avoid duplicate caching. One of the reasons for developing LCFS as a native file system was to avoid some of the disadvantages of a merged solution.

hi, I'm the author of LCFS, let me know if you have any questions about how it works or how we measured the performance.

This was done using fuse to make it portable across different distros. We have not seen any major issues testing it as a docker storage driver, compared to the graphdrivers in use today.

It is a mini-file system, working directly on top of a block device. Having something else in between is not a good idea. Much of the data handled by docker storage driver is temporary and does not even have to hit disk before thrown away.

This project is currently in experimental mode. We would appreciate any feedback you may have.

I would say image layer file system is not that hard to solve if you approach the problem correctly. It looks like all prior attempts tried to use general purpose solutions and in some cases a combination of those (for example, overlay on top of ext4) for dealing with a much smaller problem and ran into unnecessary complications. A lot of the complexities in a general purpose solution is unwarranted for a image layering file system.

We looked at the requirements of a docker storage driver and built a solution from scratch, just to address those efficiently and nothing else.

There are many problems a storage driver for docker need to solve. We are not replacing docker here, just providing a better solution for the storage driver (API of which is defined by docker) part of it. Solutions in use today are half baked and have too many problems to deal with.

Docker is already doing a good job of de-duplication by identifying shared data between applications and keeping a unique copy of shared data on disk. But many of the existing storage drivers today, would create multiple copies of such shared data in memory when many applications (containers) consume such shared data. Applications are running on top of file systems and each need a separate instance of the file system and Linux kernel does not know if all those file systems are sharing same data underneath. Each application also require a unique view of the file system and they may choose to modify their file system (thus their shadow copy of shared data) and such changes should be hidden from other applications (containers). That is where cloning technologies come into play. And there are many other things a docker storage driver need to solve. Rather than focusing on one aspect of the whole problem, it would be better to look at the whole picture and see if there are any solutions out there solving all those problems correctly and efficiently for a docker storage driver.

This is not a general purpose file system, but something purpose built for serving as a docker storage driver. LCFS outperforms existing solutions and does not have any correctness issues.

It is an experimental release at this point. We would like to receive feedback from the community and make it successful for everybody. Our tests so far look very promising.

BTRFS has so many drawbacks using it as storage driver for docker. See the bottom of the page https://docs.docker.com/engine/userguide/storagedriver/btrfs.... Primary drawback being each container creates unique copy of the data in kernel page cache and taking away system resources from applications.

Again, the focus of this blog is not about a de-duplication story, but about a new docker storage driver.

This is a custom built solution for docker storage driver. A de-duplication server cannot be docker storage driver, as there are more things involved in a docker storage driver like layer management, rapid cloning and deletion of those, efficient diffing between layers and stuff like that. The existing solutions are AUFS, Overlay, devicemapper, btrfs and zfs, and a comparison should be made against those.

We did not solve a typical de-duplication problem with LCFS. If you write two copies of the same data to LCFS, there will be two copies of data stored. We may provide such de-duplication feature at some point in future.

Docker is already doing de-duplication by keeping shared data in image layers and expect the storage driver (graphdriver) to manage shared data between many container application efficiently without instantiating multiple copies of shared data in memory and disk. That problem can be solved by well known file system snapshot and cloning technologies and that is what LCFS is doing, and a lot more efficiently than any existing storage drivers out there.

The README is updated with more details on the tests performed. Also concerns raised about setup scripts are fixed now. Thanks a lot for pointing those out.

I am not surprised on your first impression that the published results are too good to be true. But we have proved that a custom built solution can outperform half baked solutions very easily, in addition to providing correctness.

Yes, it is a native file system optimized for large number of containers (clones), sharing read only image layers. LCFS also understands the ephemeral nature of data stored in /var/lib/docker and avoids some of the complexities seen in general purpose file systems.

Yes, also it does not have data corruption issues reported with overlays (readers vs writers). Also it supports atomic rename. NO whiteouts required as in the case of a union file systems (so not to worry about deleted files coming back).