In the last few years I've transitioned to "My family just wants plex to work and I could give a shit about the details". I think I'm more of the target audience. When I had my non-truenas zfs set up I just didn't pay a lot of attention, and when something broke it was like re-learning the whole system over again.
I accidentally gave myself the opportunity to test out my automations when I decided I wanted to rename my k8s nodes (FQDN rather than just hostname). When I did that, everything broke, and I decided it would be easier to simply re-provision than to troubleshoot. I was up and running with completely rebuilt nodes in around an hour.
In my experience, a vanilla install and some daemons sprinkled on top works better than these GUI flavours.
Less breakage, fewer quirks, more secure.
YMMV and I’m not saying you’re wrong - just my experience
Similarly while docker's CLI interface is relatively nice, it's even nicer to just take my phone, open a browser, and push "update" or "restart" in a little gui to quickly get things back up & going. Or to add new services. Or whatever else I want. Sure I could SSH in from my phone, but that's awful. I could go get a laptop whenever I need to do something, but if Jellyfin or Plex or whatever is cranky and I'm already sitting on the couch, I don't want to have to get up and go find a laptop. I want to just hit "restart service" without moving.
And that's the point of things like TrueNAS or Unraid or whatever. It makes things nicer to use from more interfaces in more places.
But for probably 90% of users, they just want a UI they can click through and mostly use the defaults, then log into now and then to make sure things are good.
The UI is also especially helpful for focusing on things that matter, like setting up scrubs, notifications, etc. (though even there I think TrueNAS could do better).
It's why Synology persists, despite growing more and more hostile to their NAS owners.
I guess the thing is that I've never done that with [Free|True]NAS :). I've always used some kind of hardware (either retired rack mount servers or thin clients) and installed FreeNAS on there, and then it never really felt like it saved me a lot of time or effort compared to just doing it all manually.
Of course, I'm being a typical "Hacker News Poster" here; I will (partly) acknowledge that I am the weird one, but at the same time, as I said, if you're in the market to install TrueNAS on your own hardware, you are also probably someone who could relatively easily Google your way through doing it manually in roughly the same amount of time, at least with NixOS.
But this is one aspect of my life that I look at as core infrastructure that needs to just work, and I don’t really gain anything from rolling my own in this particular category.
I still run a mini home lab where I do all kinds of tinkering. Just not with storage.
But I also completely understand wanting to do this all manually. I’ve been there. That’s just not where I am today.
It's a little different though; TrueNAS still requires a fairly high level of technical competence to install on your own hardware; you still need to understand how to manage partitions and roughly what a Jail is and basic settings for Samba and the like. It's not completely analogous to Dropbox because Dropbox is trivial for pretty much anyone.
The problem with TrueNAS is it fills the niche where it’s targeting people who want to tinker, but don’t want to learn how to tinker. Which is likely a smaller demographic than those who are willing to roll their own and those who just want a fully off-the-shelf experience.
I also think Synology would be closer to the Dropbox experience than TrueNAS.
Sure, but both are free options, one just takes strictly less work to do the task of being a NAS. Why would I pick the one that's just more work for the same result? If I want a NAS, why would I roll my own with NixOS instead of just picking a distro that focuses on being a NAS out of the box? What's the benefits of doing it manually?
If I want to just play around with stuff in a homelab setting, that's what proxmox clustering is for :) But storage / NAS is boring. It just needs to sit there doing basic storage stuff. I want it to do the least amount of things possible because everything else depends on storage being there.
> But for probably 90% of users, they just want a UI they can click through...
90% of HN users. More like 99.99% in the real world!I think if you've gone through the effort of setting up Ansible scripts for setting up & maintaining a NAS, you probably are not actually making a NAS anymore. Like maybe you're doing Ceph or Gluster cluster or something, which can be fun to play with. Heck, I did that with a bunch of ODROID-HC2's as well. It was fun to setup a cluster storage system.
It also wasn't practical at all and at no point did it ever seriously compete with replacing my "real" NAS (which currently is Unraid, but I'd absolutely consider switching to TrueNAS in a future upgrade), since the main feature that a NAS needs to provide is uptime.
I agree for the most part, though even vanilla Ubuntu has Cockpit if you need a GUI.
Personally I find that getting it set up with NixOS is pretty straightforward and it's "set and forget", and generally it's not too hard to find configurations you can just copypaste done "correctly". And of course, if you break something you can just reboot and choose a previous generation. Of course restarting still requires SSHing in and `systemctl restart myapp`, so YMMV.
A few big shares is all I really need; I no longer create a share for every single idea/thing I can think of.
i've wanted to get a NAS running again, but while the QNAP form factor is great, the QNAP OS was overkill – difficult to manage (too many knobs and whistles) – and ultimately not reliable.
so, i'm at a junction: 1) no NAS (current state), 2) custom NAS (form factor dominates this discussion – i don't want a gaming tower), or 3) back to an off-the-shelf brand (poor experience previously).
maybe the ideal would be a Mac Mini that i could plug 4 HDDs into, but that setup would be cost-inefficient. so, it's probably a custom build w/ NixOS or an off-the-shelf, but i'm lacking the motivation to get back into the game.
ZFS on macOS sucks really bad, too, so that rules out the obvious alternative.
- External enclosure disconnected frequently (this is more of an issue with the enclosure and its chipset, but I bought a reputable one)
- Many services can't start without being logged in
- If you want to use FileVault, you'll have to input your password when you reboot
Little things went wrong too frequently that needed an attended fix.
If you go off the shelf, I recommend Synology, but make sure you get an Intel model with QSV if you plan to transcode video. You can also install Synology OS to your own hardware using Xpenology - its surprisingly stable, moreso than the mac mini was for me.
How about if you _don't_ plan to transcode video? For example, this years models DS425+ uses the (six year old) Intel Celeron J4125, while the DS925+ uses the (seven year old) AMD Ryzen Embedded V1500B. Why choose one over the other?
They can be had for a bit less than the Mac mini, and I had no issues getting headless Linux working on there. I even have hardware transcoding in Jellyfin working with VAAPI. I think it cost me about $400.
Not to jinx it, but I have never had a hard drive failure since around 2008!
I have one enclosure plugged into a USB 3.0 line, another plugged into a "super speed" line, and one plugged into a Thunderbolt line (shared with my 10GbE Thunderbolt card with a 2x20 gigabit splitter).
This was deliberate, each is actually on a separate USB controller. Assuming I'm bottlenecked by the slowest, I'm limited to 5 gigabits per RAID, but for spinners that's really not that bad.
ETA: It's just a soft RAID with ZFS, I set up each 8-bay enclosure with its own RAIDZ2, and then glued all three together into one big pool mounted at `/tank`. I had to do a bit of systemd chicanery with NixOS to make sure it mounted after the USB stuff started, but that was like ten lines of config I do exactly once, so it wasn't a big deal.
Though even flaky adapters just tend to lock up, I think.
But often after extensive use, `dmesg` would complain about problems when talking to the drives, e.g. drive not responding or other strange error messages (forgot the exact text but very irritating and google-fu didn't help). There were also problems with SMART commands passthrough and drive power management e.g. sleep/standby adjustment which wasn't reliable when talking via bridge chips.
I use only disks directly connected to SATA controllers afterwards and no such issues ever happened again.
ETA: It would certainly take longer with more data. I've not gotten anywhere close to the 288TB
?!?
How do you fill 288 TB? Is it mostly media?
>I liked FreeNAS for awhile, but after a certain point I kind of just learned how to properly use Samba and NFS and ZFS, and after that I kind of felt like it was just getting in the way.
I've been a mostly happy TrueNAS user for about four years, but I'm starting to feel this way.
I recently wrote about expanding my 4-disk raidz1 pool to a 6-disk raidz2 pool.[1] I did everything using ZFS command-line tools because what I wanted wasn't possible through the TrueNAS UI.
A developer from iXsystems (the company that maintains TrueNAS) read my post and told me that creating a ZFS pool from the zfs command-line utility is not supported, and so I may hit bugs when I use the pool in TrueNAS.
I was really surprised that TrueNAS can't just accept whatever the state of the ZFS pool is. It feels like an overreach that TrueNAS expects to manage all ZFS interactions.
I'm converting more of my infrastructure to NixOS, and I know a lot of people just manage their NAS with NixOS, which is sounding more and more appealing to me.
[1] https://mtlynch.io/raidz1-to-raidz2/
[2] https://www.reddit.com/r/truenas/comments/1m7b5e0/migrating_...
I kind of purposefully don't fill it up :).
This has been a project of tech hoarding over the last ~8 years, but basically I wanted "infinite storage". I wanted to be able to do pretty much any project and know that no matter how crazy I am, I'll have enough space for it. Thus far, even with all my media and AI models and stock data and whatnot, I'm sitting around ~45TB.
On the off chance that I do start running low on space, there's plenty of stuff I can delete if I need to, but of course I probably won't need to for the foreseeable future.
> I'm converting more of my infrastructure to NixOS, and I know a lot of people just manage their NAS with NixOS, which is sounding more and more appealing to me.
Yeah, that's what I do, I have my NixOS run a Samba on my RAID, and it works fine. It was like fifteen lines of config, version controlled, and I haven't thought about it in months.
older nas was 4x 4tb drives when I retired it and passed it to a friend.
Current nas is a 6-bay synology with ram and nvme upgraded, along with a 5-bay expansion. All with 12tb drives in a 6drive 2-parity and 5-drive 2-parity arrays. I'm just under 40tb used mostly media.
If you do a vanilla FreeBSD+samba+NFS+ZFS setup, you'll need to edit several files around the file system, which are easy to forget months down the line in case of adjustment or disaster recovery.
Now I just run Ubuntu/Samba and use KVM and docker for anything that doesn't need access to the underlying hardware.
It actually has been considerably more reliable than the MediaSonics than they replaced.
I'm looking to retire my power hungry 4790k server and go with a mini pc + external storage.
I wouldn't do USB-connected enclosures for NAS drives - either SATA via SFF, or Thunderbolt.
The cost is very low, but it would have to be a nvme only build.
“Have to”, since I no longer suggest virtualizing TrueNAS even with PCI passthru. I will say the same about zfs-over-USB, but you do you. I’ve had too many bad experiences with both (for those not on the weeds here, both are officially very much not supported and recommended, but they _do_ work).
I really like the TrueNAS value prop - it makes something I’m clearly capable of by hand much easier and less tedious. I back up both my primary zfs tank and well as my PBS storage to it, plus cold backups. It does scheduling, alerts, configuration, and shares, and nothing else. I never got the weird K8s mini cluster they ship - seems like a weird thing that clashes with the core philosophy of just offering a NAS OS.
At first, I thought I might just go with TrueNAS. It can manage my containers and my storage. But it's got proprietary bits, and I don't necessarily want to be locked into their way of managing containers.
Then my plan was to run Proxmox with a TrueNAS VM managing a ZFS raidz volume, so I could use whatever I want for container management (I'm going with Podman)
But the more I've researched and planned out this migration, the more I realize that it's pretty easy to do all the stuff I want from TrueNAS, by myself. Setting up ZFS scrubbing and SMART checks, and email alerts when something fishy happens, is pretty easy.
I'm beginning to really understand the UNIX "do one thing and do it well" philosophy.
It's actually not too bad; the main "server" idles at around 14W and the power supply for it only goes to 100W under load. The drive bays go up to 100W (I think) but generally idle around 20W each. All together it idles at around ~70-80W.
Not that impressive BUT it replaced a big rack mount server that idled at about 250W and would go up to a kilowatt under load.
> Should you care about the "USB bottleneck"?
Not really, at least not for what I'm doing. I generally can get pretty decent speeds and I think network is often the bottleneck more than the drives themselves.
> How do you organize this thing so it's not a mess of USB cables?
I don't :). It's a big mess of USB cables that's hidden in a closet. It doesn't look pretty at all.
4k Blu-ray rips peak at over 100 Mbps, but usually average around 80 Mbps. I don't know how much disk I/O a Minecraft server does ... I wouldn't think it would do all that much. USB2 (high-speed) bandwidth should be plenty for that; although filling the array and scrubbing/resilvering would be painful.
That said, I have lots of home movies that just so happen to be at the exact same bitrates as Blu-rays and after the initial setup, I've never really had any issues with them choking or any bandwidth weirdness. Minecraft doesn't use a ton of disk IO, especially since it is rare that anyone plays on my server other than me.
I do occasionally do stuff that requires decent bandwidth though, enough to saturate a WiFi connection at the very least, and the USB3 + USB SS + Thunderbolt never seems to have much of an issue getting to Wifi speeds.
I liked FreeNAS for awhile, but after a certain point I kind of just learned how to properly use Samba and NFS and ZFS, and after that I kind of felt like it was just getting in the way.
Nowadays, my "NAS" is one of those little "mini gaming PCs" you an buy on Amazon for around ~$400, and I have three 8-bay USB hard drive enclosures, each filled with 16TB drives all with ZFS. I lose six drives to the RAID, so total storage is about ~288TB, but even though it's USB it's actually pretty fast; fast enough for what I need to for anyway, which is to watch videos off Jellyfin or host a Minecraft server.
I am not 100% sure who TrueNAS is really for, at least in the "install it yourself" sense; if you know enough about how to install something like TrueNAS, you probably don't really need it...