That’s a neat way of visualizing it.

It is interesting that the “remainder” chunk of the drivers is so big. Wonder if they need some more categories.

I was expecting a block chart, kind of disappointed it isn't.

Wow, I'm surprised by how small the file systems are.

On the smaller side, Ext4 at just 21,515 lines, and on the larger side, Btrfs at 55,758 lines.

It almost makes me think that writing a new file system in Rust, might actually be a tractable project.

Oh please do, because what really screws up filesystem implementations is the language they're written in.

It's a good thing that you warned them off, or they might've accidentally gone and done something interesting or enjoyable.

I think it's a great project to tackle.

Here are my criticisms :

1) LoC doesn't indicate complexity. When you're looking at a code base and you see that it's 65000 lines, that by no means should be an indicator of whether or not a re-write is easily feasible -- especially with a filesystem.

2) Language shouldn't be a motivator for a re-write without considering why the language offers advantages, the end users don't care about which file extension is littered across the source tree.

Both of those tendencies (one to re-write everything in your favorite language, and two to come across with an idea that you understand how complex a mechanism is simply from LoC) indicate to me, anecdotally of course, a certain lack of experience; and I have a hard time imagining someone with those concerns tackling a re-write of a filesystem that gains 'the audience' any benefits.

If it's just for the sake of personal toys and fooling around with your own machine I feel as if those motivations are fine, but I have concerns if 'a new filesystem in Rust' comes from what I view as shaky initial motivations.

Yep, because everyone who is going to do something interesting or enjoyable starts with the question "should I use language X because it's cool?".

Look, I'm as into cool FS-hacks as anyone, but writing a new filesystem has 99+ problems and the language used is either the last of them, or it isn't even in the set.

Granted, you could (on Linux) write a little pseudo-filesystem (e.g. /proc or /sys) and not face any of the issues involved in extN or btrfs etc. I didn't get the sense that this was what the OP intended, but I could be wrong.

Writing a basic filesystem using FUSE is pretty simple, fun, and can occasionally be useful.

I frankly don't understand why he doesn't realize that the borrow checker is just the thing to prevent all the bugs you can have with a file system implementation.

All the bugs? I doubt that.

Many of the worst bugs (data loss and corruption)? Probably.

Having something like the Rust borrow checker around to make sure you don't make dumb mistakes is handy, to be sure. But, let's not oversell it as some kind of panacea. I bet I could write a FS in Rust with all kinds of bugs, and the borrow checker wouldn't be able to stop me.

I also thought that'd be a fun project. BTRFS seems to have the largest feature set although many features are experimental but maybe there would be a place for something smaller that scales well and maybe has some of the new features like deduplication

I'm writing a hobby kernel/operating system and one of the things that surprised me was how simple ext2 is. It really isn't much different from fat32 in many ways, just a bit more flexible and designed in a more future-proof way.

Obviously ext2 is a very outdated filesystem by today's standards, but considering ext3 and ext4 are essentially just some extensions slapped onto ext2, I still didn't expect it.

Another thing I found really impressive was just how resilient e2fsck is. During early development of my ext2 code I frequently mangled the file system in a myriad of weird ways, but no matter how much I broke things, e2fsck was almost always able to restore the file system into a reasonable state.

(For the curious, there is some excellent documentation on the ext2 file system at https://www.nongnu.org/ext2-doc/ext2.html that I mostly used for my implementation of it. Sadly no such thing seems to really exist for ext3/ext4).

Filesystems tend not to be all that complex to implement.

Simple filesystems - those oriented at speed, without integrity checking, compression, encryption and raid-like features. Which are extremely useful and desirable in a filesystem.

When people tried to do a full-featured filesystem, with a remarkable exception (ZFS) it's either a disappointing unfinished mess (btrfs) or a promising, but a long-term project taking years (bcachefs).

Making a filesystem with strong support of important features is an extremely demanding project.

Don't forget: buggy filesystem code can easily lead to catastrophic data loss, which ruins your filesystem's reputation.

Lines of code isn't the only important metric. A project with a smaller number of LoC can take a long time to fully test and debug and validate.

Hammer2 comes to mind as well..

Does it run on Linux or ordinary BSD? Does any company use it?

I'm sorry, as I keep hearing these criticisms around BTRFS, I still don't know why. What is so messy about it? Honest question

Have you tried reading the code? It's twice as large (107KLOC) as ext4 (45KLOC), where ext4 is viewed as a natural evolution of ext2 (7KLOC).

45KLOC of C code already sounds really scary to me, although yeah, journaling is really nice to have.

Yes, that's scary. Understandable, but it does come with a lot of functionality that's nice to have no?

It's "perpetually half-finished". Bad track record with reliability and loss of data, lots of stories on the internet. It perhaps got better, but RAID5/RAID6 for metadata (standard use of disks) still not oficially recommended and thus standard RAID5/RAID6 is not reliable. Also, look at some FS benchmarks - BTRFS is still much slower than EXT/XFS, and slower than ZFS.

About the only class of scenarious to use BTRFS instead of ZFS is if you want the features, do not need RAID5/6, require support by Linux distributor/kernel developers or for some other (hard to think of) reason you can't use ZFS.

Thanks, that makes sense. I should look more into it

No problem. For more, this looks like a good intro

https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...

Filesystems aren't complex to implement -- even complex filesystems.

What's harder is making them as robust as they need to be for practical daily use. Doing that is a matter of comprehensive testing, though. With filesystems, the hard part is QA, not implementation.

Oh it's word games with "implement". OK then, let's agree then developing a modern filesystem into a robust software for practical daily use is hard.

The difficulty in filesystems is generally getting to the point of high enough reliability that people trust then with their data. This is why development and adoption of new ones tends to be slow, not the quantity or complexity of implementation. So for a hobby it's quite accessible because that expectation of reliability isn't there.

Wow arch/arm (which is 32bit) is bigger than x86 and powerpc which have 32/64 under the same subsystem. I guess it's mostly device tree bindings?

For anyone wondering, drivers are 5205102 out of 7616785 lines, or about 68.34% of all lines.