Great job. How does it perform? I thought ZFS needed tons of RAM.

Aside from the "needs many gigs of RAM, even without deduplication turned on" angle, the lack of ECC memory would prevent me from using ZFS on raspi.

Sooner or later you will trash your data unless you're using ECC memory (which the Raspberry Pi RAM chips are not): http://louwrentius.com/please-use-zfs-with-ecc-memory.html

For the record, I also think it's cool to get this compiled and running on RasPi; just wanted to add the ECC info for anyone now thinking of doing something with this in production :-D.

ZFS without ECC is no worse than any other file system which doesn't do checksumming.

See a quote from Matt(hew) Ahrens (zfs developer)[0]:

  There's nothing special about ZFS that requires/encourages the use
  of ECC RAM more so than any other filesystem. [...] I would simply
  say: if you love your data, use ECC RAM. Additionally, use a
  filesystem that checksums your data, such as ZFS.

[0] http://arstechnica.com/civis/viewtopic.php?f=2&t=1235679&p=2...

The Raspberry-Pi can't perform that good because it lacks USB-3 or something else with good performance (it only has USB-2)... It is not a very good choice for a file server. The ODROID-XU4 seems like a much better choice even if it is double the price.

1G is not a lot of RAM?

Apparently, ZFS needs 1GB of RAM per 1TB of storage.

You're probably thinking of deduplication (whose memory requirements is ~320 bytes per deduplicated block, and which generally requires way more than 1GB per TB, it's highly variable depending on storage load but I've seen quotes from 5 to 20GB/TB). Most users don't benefit from deduplication and it's usually recommended not to enable it unless you do demonstrably get large benefits from it (on FreeBSD, `zdb -S` will simulate and estimate deduplication benefits).

1GB is the recommended conf for ZFS on FreeBSD, it will benefit from more (and may need some tuning below 2GB) but even that's not a requirement, the FreeBSD wiki's ZFS tuning page quotes "ZFS running nicely on a laptop with 768 Megs of physical RAM" with tuning (and provides configuration values).

I've always been wondering if file system RAM caches couldn't be moved to a fast SSD nowadays. Do you think this would be possible with, say, an Intel 750?

ZFS (and probably others) let you do that, kinda: it always has a primary in-memory cache but you can add level 2 cache devices (L2ARC) to the pool. In fact you can extend two different caches to storage devices: L2ARC is a read cache, and you can add a separate set of Log Devices used by the ZFS Intent Log, where writes are cached.

I'm pretty sure that is already a thing. Are you talking about something like L2ARC or swap disks?

This model only has 512MB, minus 16MB for the video system.

While I haven't tried crochet, I have tried building my own RPi-B+ image. You mention in your guide that you'll post updates "when you have something", and judging by your post it seems you have.

Since I had difficulties compiling the zfs (and one other, which I don't recall) module, I'm wondering if you'd care to share your experience in your guide?

Kudos to your very well thoughtout and thorough guide.

I got it going a few days ago, have been meaning to update the guide and post some images so people can skip the compilation. Star the repo for updates.

You're probably thinking of the opensolaris module. There's a small patch to apply and some KERNCONF to trigger the module build but otherwise its straightforward -- once you know where to look!

Give crochet a try. I've used it recently to build a FreeBSD 11 image for RPi B+ recently and it just works.

Thanks. Yes it did build just fine. Now I just have to figure out what's "stalling" during boot.

It compiled with zfs and all fine though :)

Interesting, I thought ZFS required a 64-bit system. Is this no longer the case? (I am using UFS on powerpc32 since it's not supported for 10.2.)

64-bit is very strongly recommended, but it does work on 32-bit with tuning: https://wiki.freebsd.org/ZFSTuningGuide

Nice work!