News for hackers. Stuff that matters.
Re:The latest DSM update makes btrfs drives unavailable on b... by justapassenger on Wednesday April 14, 2021 @12:46AM (#26803038)
Wow, this beyond unacceptable and borderline illegal. Even tho I’m very happy owner of Synology, my next NAS won’t be it for sure. What are good alternatives for home usage? QNAP?
Re:The latest DSM update makes btrfs drives unavailable on b... by KozmoNau7 on Wednesday April 14, 2021 @03:46AM (#26804178)
I built a DIY NAS in a small cube case (Fractal Node 304), which can house a mini-ITX motherboard and six 3.5" drives, and has pretty good (and quiet!) cooling for a case that size (two 92mm fans in front and a 140mm in back). The drive bays are not hot swappable, but are very easy to replace when the top is taken off the case.

For the OS, I put openSUSE on it, set up a btrfs pool and configured SMB, NFS, FTP and SSH/SCP manually. It also functions as my HTPC, with Kodi as the frontend.

The Celeron J4105 is passively cooled and rarely sees 50C even under full load, plus it has hardware video decoding, so it can play back 4K video all day with no strain.

It's not a turnkey solution like a Synology or QNAP box, but it's a lot more flexible.

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Friday April 16, 2021 @05:24AM (#26831558)
Interesting. What is roughly the total price, the average power consumption, and what is the main advantage compared to ARM systems? Am I correct to assume the Intel chips still draw a lot more power than the single-digit Watt numbers of ARM systems, which would become a substantial accumulated cost over the years?
Re:The latest DSM update makes btrfs drives unavailable on b... by KozmoNau7 on Friday April 16, 2021 @09:25AM (#26833572)
The price for the hardware was DKK 2,500 or $400. Synology doesn't have a 3.5" 6-bay NAS, but their cheapest 4-bay is the DS420j at DKK 2300, or around $370. So while that is slightly less expensive and probably uses less power, it also has less room for disks and is significantly less flexible. If you have modest demands and just need something to store your files, it'll work.

Or you could go for the DS1612+ to have room for 6 disks like in my build, which will set you back DKK 6,900 or $1,100. And that uses an AMD Ryzen CPU, not ARM. On the upside, it also has transcoding and media playback functionality like my build, and some features my build doesn't have, like an upgrade to 10gbit ethernet.

I'm not counting disks in these prices, since I used disks I already had, which I would also have done if I had bought a prebuilt NAS.

The TDP for the Celeron J4105 in my build is 10W, representing "the average power, in watts, the processor dissipates when operating at Base Frequency with all cores active under an Intel-defined, high-complexity workload". It very rarely runs at anything approaching a high-complexity load, even when transferring files over gigabit ethernet. It idles at around 30-35C passively cooled aside from the case fans, which run inaudibly at 500rpm.

Obviously the chipset and RAM also draw some power, as do the disks and fans and so on, but it compares very favorably to the quoted 51W under load/25W idle for the DS1612+. That is why I specifically chose a motherboard with that class of CPU.

Let's say the 51W figure is the same for my build and assume a worst-case load, running at full power 24/7. With the current electricity prices here, that's DKK 773 or $125/year.

Compare to the DS420j, which is quoted at 22W, DKK 335 or $54/year, for a device that is significantly less capable.

And my build is not just a NAS. It also serves as my HTPC and couch gaming machine for retro and emulated games, among other things. Plus it's 100% under my control, I get to decide what's installed and which features are enabled/disabled. For me, the choice is obvious.

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Wednesday April 14, 2021 @10:38AM (#26807692)
> It's not a turnkey solution like a Synology or QNAP box, but it's a lot more flexible.

That is not even the biggest advantage. The biggest advantage, in my opinion, is that you will always get updates and can run that for an unlimited time, without breaking your setup.

Re:The latest DSM update makes btrfs drives unavailable on b... by justapassenger on Wednesday April 14, 2021 @11:33AM (#26808511)
Building your own is a cool thing, and gives you many benefits. But updates and Linux are very prone to breakages - there’s too much hw and sw combinations to expect solid experience.
Re:The latest DSM update makes btrfs drives unavailable on b... by KozmoNau7 on Wednesday April 14, 2021 @12:15PM (#26809165)
That is why I chose very commodity hardware and run openSUSE Leap 15.2 on it. I wanted a solid distro with a reasonable update policy, not something that pushes 100+ new updates every week. Debian Stable or perhaps Testing would also have been good choices, but I like YaST.

In the interest of full disclosure, I have had one issue with my build. The motherboard has four SATA connectors; Two from the Intel chipset, which work flawlessly, and two from an additional ASM1061 chipset, which don't. They seem to work initially, but when you start pushing larger amounts of data through them, errors start to pop up.

As far as I can tell, it's a either a driver issue or a BIOS issue, I can't remember whether I got those same errors on an ASM1061-based 2-port SATA PCIe expansion card or not. Either way, I replaced it with Marvell-based 4-port card instead, and just avoid using the ASM1061 ports for now.

Next time I add a disk to the system, I'll connect it to the potentially troublesome ports and run some tests, to see whether it was a driver issue that has been cleared up.

So yeah, not perfect, but good enough for me :-)

Re:The latest DSM update makes btrfs drives unavailable on b... by pnutjam on Wednesday April 14, 2021 @12:43PM (#26809586)
My home "NAS" running opensuse has been continually upgraded since something like OpenSuse 12. I have data drives and I have an OS ssd that's only 50GB. It's rock solid.
Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Thursday April 15, 2021 @06:53AM (#26818807)
> But updates and Linux are very prone to breakages - there’s too much hw and sw combinations to expect solid experience.

That seems like FUD to me. I am running Linux since over 20 years. A NAS's hardware interfaces are mostly HDD and network drivers, and I have not seen them breaking in many years.

And especially for PC-platform hardware, or ARM SoC in which Linux is the dominant OS. I could understand if NAS vendors would say that in their advertisement, but what is the substance of claims like this? Especially since these things runs Linux as well.

Re:The latest DSM update makes btrfs drives unavailable on b... by vetinari on Thursday April 15, 2021 @01:23PM (#26823464)
There are way more things that can go wrong, not just drivers.

I've run a ThinkServe machine with CentOS and ZFS for several years. Kernel and ZFS updates would break it regularly, requiring tinkering to make it work again (my favorite: zfs#8885).

Nowadays I prefer when somebody else did the integration work.

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Friday April 16, 2021 @05:14AM (#26831485)
Well, I see better what point of view this is coming from.

However, ZEF is not part of the standard kernel, IIRC there are still license issues and it needs to be built separately, which is not necessary with any normal file system. I enjoy some amount of tinkering, too, but I believe it is expecting too much that everything works out of the box with non-standard kernel modules.

In the same vein, I also do not understand entirely why people want to have something still experimental like btrfs precisely on their NAS, and apparently use that without an extra load of backups. btrfs has many qualities, but it is still not considered as robust as ext4 (the current standard Linux file system), and for a long time it had practically no recovery option if file system data became corrupt. For me, it is very clearly not the first choice for a NAS or server which should run completely hassle-free. IMO, everything one needs for that is on Debian stable, so why not use that? And why in heaven should one make oneself dependent on what a specific hardware vendor does?

Re:The latest DSM update makes btrfs drives unavailable on b... by vetinari on Friday April 16, 2021 @08:22AM (#26832977)
The nice think about RHEL/CentOS is, that their kernels have stable ABIs (kABI). That allows for third parties to provide binary kernel modules, and the OpenZFS project actually provides the kABI-kmod tracking repo, so you can have an installation without any developer tools installed (which I prefer). So the problem wasn't really with kernel modules, the problem was the integration with the system, and nobody really cared - the above mentioned issue is still open, 2 years later.

Another nice thing about CentOS is, that it was supported for 10 years. Debian is supported only for 3. It tkaes 5 years for Synology to get a new release done, so they also do have kinda-long term support.

Re:The latest DSM update makes btrfs drives unavailable on b... by KozmoNau7 on Friday April 16, 2021 @08:41AM (#26833151)
Btrfs is stable and has been widely used in production for years. Facebook uses it, SUSE CaaS uses it, the officially recommended platform for SAP is SLES, which has defaulted to btrfs for the root filesystem since 2014.

Btrfs brings snapshots, rollback, deduplication, storage pools, checksumming, send/receive and a whole host of other features. It's a completely different type of filesystem to ext4.

Calling btrfs "experimental" at this point is like calling electric cars "experimental".

Re:The latest DSM update makes btrfs drives unavailable on b... by muxator on Wednesday April 14, 2021 @04:08AM (#26804323)
I whish I could do the same with a CPU that supports ECC ram. Unfortunately, a fanless CPU with ECC support (the Atom x6000 for example) is almost a chimera, or really really expensive.
Re:The latest DSM update makes btrfs drives unavailable on b... by parshimers on Wednesday April 14, 2021 @05:19AM (#26804720)
I built a custom home server using Mini-ITX form factor parts and I would have to agree with this conclusion. ECC ram in that form factor is very expensive. The biggest issue I had was that fanless ATX power supplies are not reliable. There was also a hardware bug that was bricking Atom C2xxx processors.

I think that if form factor is not an issue, one of the i3's that support ECC ram are low enough TDP to be cooled passively. Especially if it is not expected to run at full load for long periods. However I simply switched to using a big case with low RPM, good quality FDB fans. It hasn't been a problem since, and is much less cramped when I do want to tinker with it.

Re:The latest DSM update makes btrfs drives unavailable on b... by KozmoNau7 on Wednesday April 14, 2021 @06:24AM (#26805108)
I would absolutely go for a micro-ATX or even full ATX board if that allows you better choices in hardware to get ECC or other features.

My goal was to build something not too much larger than a 4-6 disk NAS, that would also function as a HTPC, sitting under my TV. That meant small size and quiet operation were the main priorities. As none of the popular consumer NAS devices use ECC RAM, I reasoned that I would do without it in my build, too.

The fans are running on the lowest variable setting controlled by the BIOS, it's whisper-quiet even with your ear next to it, but I've got some even quieter fans from be quiet, that I'm planning to swap in. The PSU is also from be quiet!, and with the placement deep in the case, with the 120mm fan oriented downwards, it's completely inaudible.

Had I not been constrained by space and noise concerns in my small apartment, I would have build a larger, more powerful and ECC-equipped machine to be hidden away in an equipment rack in the basement.

Re:The latest DSM update makes btrfs drives unavailable on b... by pnutjam on Wednesday April 14, 2021 @12:42PM (#26809565)
Bingo, this is what I recommend. It's much more flexible.
Re:The latest DSM update makes btrfs drives unavailable on b... by mfkp on Wednesday April 14, 2021 @03:15AM (#26803959)
I switched from Synology to QNAP about 5 years ago (after my Synology NAS decided to stop reading disks using their proprietary RAID, and Synology support said "sorry, we don't know how to fix it". Lost 8tb of data that day).

Two lessons learned: 1) Don't use proprietary raid, 2) Don't ever use Synology devices.

QNAP has been great though, I've been very happy so far and zero issues.

Re:The latest DSM update makes btrfs drives unavailable on b... by vetinari on Wednesday April 14, 2021 @03:31AM (#26804073)
Synology uses bog-standard mdraid. You can read the volumes on any linux machine.
Re:The latest DSM update makes btrfs drives unavailable on b... by mfkp on Wednesday April 14, 2021 @03:42AM (#26804151)
Maybe I'll have to take another look at it when I have some free time, although I vaguely remember something not working right with the standard mdraid tools. According to this post, it seems to be a slightly customized version: https://serverfault.com/questions/568166/how-to-recover-an-m...

Thinking back on this situation made me remember the terrible thermals in that old NAS. I had to replace the (fanless) PSU 2 times because it was always blowing up and creating sparks/smoke. Lol good times.

Re:The latest DSM update makes btrfs drives unavailable on b... by vetinari on Wednesday April 14, 2021 @04:00AM (#26804270)
They have it customized, they have also btrfs integrated with md, which is not in the stock kernel, but these are runtime things. Not to say there are not a ways to shoot yourself in the foot (e.g. being careful when creating ext4 volume with external machine, target nas might have older kernel that doesn't support half of the newer ext4 features and 64 bits).

I never had any problem with thermals or psu, but I've heard other people had psu problems.

Re:The latest DSM update makes btrfs drives unavailable on b... by BuildTheRobots on Wednesday April 14, 2021 @08:48AM (#26806309)
On their older models, the webUI did something funky with LUKS encryption passwords meaning you couldn't actually just mount the disks in another Linux computer as you coukd never unlock them. I have memories of us having to do the encryption setup and unlock via ssh so we could mount the drives in something else in the event of a failure (though we retired that box over 5 years ago so my memory might be rusty).

edit: my memory _is_ rusty; it was a QNAP box we had to do that with not Synology.

Re:The latest DSM update makes btrfs drives unavailable on b... by pilsetnieks on Wednesday April 14, 2021 @06:17AM (#26805073)
Last I looked into it, QNAP had some serious outstanding security issues that they weren't acknowledging and weren't fixing, that turned me off of them fast. If Synology isn't to be trusted (this plus starting to lock down part support like allowing only their own HDDs,) the only thing that remains seems to be building your own and running TrueNAS or Unraid.
Re:The latest DSM update makes btrfs drives unavailable on b... by mbirth on Thursday April 15, 2021 @10:18AM (#26820807)
I’ve come to the same conclusion. Once my DS415+ dies, I’ll build one on my own. Synology won’t see another dime from me.
Re:The latest DSM update makes btrfs drives unavailable on b... by Saris on Wednesday April 14, 2021 @01:21PM (#26810178)
DIY is the best route. Anything proprietary is going to end in disaster when they just decide they don't feel like supporting it anymore.

It can be cheaper than off the shelf NAS units too.

Re:The latest DSM update makes btrfs drives unavailable on b... by lostapathy on Wednesday April 14, 2021 @01:17AM (#26803202)
QNAP isnt as popular as synology, but I manage several from 2 to ~20 disks and they gave been great.
Re:The latest DSM update makes btrfs drives unavailable on b... by JeremyNT on Wednesday April 14, 2021 @08:05AM (#26805914)
I have always used Linux PCs. The tricky thing is finding a good small case that can support enough drives without breaking the bank. Currently I have an old Haswell i3 in a MiniITX Fractal Design node 304 case - it's fine, but it only supports 4 drives.
Re:The latest DSM update makes btrfs drives unavailable on b... by layoutIfNeeded on Wednesday April 14, 2021 @11:07AM (#26808148)
I'm also running a Debian-based NAS at home. If you want to expand beyond 4 drives I can recommend the 8-bay U-NAS chassis: https://www.u-nas.com/xcart/cart.php?target=product&product_...

Same Mini-ITX form factor, 8 hot-swappable drives, pretty solid build qualtiy in my experience.

Re:The latest DSM update makes btrfs drives unavailable on b... by justapassenger on Wednesday April 14, 2021 @11:01AM (#26808084)
So did I, until I’ve decided I’m too old for dealing with it. I’ve been working professionally with Linux systems for 20 years and counting, and finally decided that at home I just want an appliance that just works, even if it doesn’t give me all the features/flexibility I’d want.
Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Thursday April 15, 2021 @12:11PM (#26822317)
I have been working professionally with Linux since 22 years, and I have tried the same with a NAS - and found out that it has just so many annoying quirks and shortcoming that I am definitively far better off, and waste far less time if I am just using standard Debian on standard hardware. No need to work around the shortcomings of a prehistoric NFS implementation or things like that... no need to set up things from scratch because some hardware breaks or there are no updates for the system it runs.
Re:The latest DSM update makes btrfs drives unavailable on b... by ValentineC on Wednesday April 14, 2021 @12:25PM (#26809307)
Doesn't the Node 304 support up to six 3.5" drives? I have one myself.
Re:The latest DSM update makes btrfs drives unavailable on b... by katbyte on Wednesday April 14, 2021 @02:53AM (#26803810)
unraid seems to be popular, it’s where I’d look but I’m Not quite ready to jump off the syno ship yet.
Re:The latest DSM update makes btrfs drives unavailable on b... by lmm on Wednesday April 14, 2021 @02:59AM (#26803863)
I've been very happy with my Asustor so far, although I don't have anything to compare it to.
Re:The latest DSM update makes btrfs drives unavailable on b... by infogulch on Wednesday April 14, 2021 @12:57AM (#26803090)
TrueNAS?
Re:The latest DSM update makes btrfs drives unavailable on b... by acidbaseextract on Wednesday April 14, 2021 @02:33AM (#26803671)
I built a 6 disk TrueNAS Raid-Z2 box two years ago and though it was inconvenient to build vs buying something off the shelf, TrueNAS has been excellent. Does what it says on the tin and continually improving with each release.

Their cloud sync and UPS functionality have been great (flaky power around here). Haven't dug into the TrueNAS' ZFS pool/snapshot management and jails management (FreeBSD's containers) functionality, but both are compelling once I get time and a good project for them. The NAS performing so well has me itching to upgrade to 10 gigabit ethernet around the house once prices come down a bit.

Also I ran the OS off mirrored thumb drives until recently. That was pretty nifty.

Re:The latest DSM update makes btrfs drives unavailable on b... by mwambua on Wednesday April 14, 2021 @02:38AM (#26803716)
I've been thinking about going down the TrueNAS route (the TrueNAS Mini series looks really nice), but it seems a little expensive. Is the extra cost worth the convenience of having a machine that you know will stay compatible with FreeNAS?
Re:The latest DSM update makes btrfs drives unavailable on b... by acidbaseextract on Wednesday April 14, 2021 @02:48AM (#26803779)
Here's a detailed review that answers that question: https://www.servethehome.com/ixsystems-truenas-mini-x-zfs-na... (the video linked there is also good)

On page 5 they tally it up as a $400 difference vs building your own.

Re:The latest DSM update makes btrfs drives unavailable on b... by infogulch on Wednesday April 14, 2021 @10:37AM (#26807680)
A custom case, labeled drive cages (with software identification), integrated and tested component bom, 1 year system warranty... for a $400 premium (that's 8 hours if I value my time at $50/hr). Worth imo.
Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Wednesday April 14, 2021 @05:32AM (#26804794)
You can use a Raspberry Pi (perhaps a Pi 400) if you work around the fact that the internal flash memory is not made for constant writing. Thus, you have to arrange the system so that it is read-only in normal state - which is not difficult. The Pi 400 also has an advantage that it has a keyboard, so you can cleanly shut it down without logging in, which makes a lot of sense for a storage device.

Also, for RAID1 you need an extra powered USB hub. The normal power supply of the Pi 400 is not sufficient for more than one disk.

Then just install a software RAID, NFS and possibly rsync and Samba. A simple NAS built in this way will come at about 240€, depending on disk size and will be mostly equivalent to a commercial one for twice the price. As an advantage, you can run well-known, standard software which is completely under your control. And since it is Debian-based Raspbian, it can be upgraded easily without breaking stuff.

Re:The latest DSM update makes btrfs drives unavailable on b... by KaiserPro on Wednesday April 14, 2021 @06:36AM (#26805213)
what ever you do, dont do this.

The Pi400 is a brilliant machine, but this is not a way to make networked storage for data you want to keep.

if you want to use a raspberry pi for network storage, and you want to keep your data, make sure you have n+1 pi & disk machine.

Raid isn't going to help you here, you need redundant nodes. for the same €240 you could get two minimum spec pi4s PSU and usb HDDs

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Wednesday April 14, 2021 @08:18AM (#26806054)
> make sure you have n+1 pi & disk machine.

Why? The thing that could fail are the disks. So, one can use a RAID1 with two disks and this should, for normal applications be enough. I had a DNS-232 before as a NAS, it had 32 MB RAM, a tiny ARM CPU, and two large disks. It was working nicely over about ten years, and one could even run a web server plus a (slow) wiki on it (I had to replace the casing once, for a botched firmware update). The main problem I had with it that it was that it was impossible to update (I was running Alt-F firmware, which was developed by the community after the original firmware had constant problems with unneeded RAID synchronization, and the vendor had to release its code due to being bound to the GPL).

The Pi400 is wildly more powerful, has a much faster CPU which only draws a few Watt, and can easily be connected to several USB disks.

> Raid isn't going to help you here, you need redundant nodes.

Why exactly? My set-up is working well.... did I miss it needs to be somehow lighter than air (because heavier-than air flying machines are not possible, y'know)?

Re:The latest DSM update makes btrfs drives unavailable on b... by KaiserPro on Wednesday April 14, 2021 @09:23AM (#26806708)
> Why exactly? My set-up is working well

congratulations I am very happy for you!

The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much.

You can use device IDs, which gives better results. Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile.

> Raid isn't going to help you here, you need redundant nodes.

This setup will have four common failure modes:

o power issue leading to corrupt "/" or any other SD card failure

o usb disk breaking

o OS crash borking everything

o Accidental deletion

Raid might protect against a usbdisk breaking, assuming you spot it in time (most people don't have monitoring.) its not even going to give you a performance boost either, its USB3.

This is why n+1 is better. assuming you're not keeping each node next to each other then you are much less likely to experience a simultaneous outage.

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Wednesday April 14, 2021 @10:02AM (#26807204)
> firstly you are reliant on the disks presenting themselves in the same order at the same time.

In Linux, which is what Raspbian is, it is standard nowadays to use UUID disk labels.

> Another issue I've bumped into is that mdadm has come up before the USB disk is initialised, causing all sorts of issues. in short raid over USB is just plain fragile.

No issue with that. I guess systemd handles this. Only thing I found is that the hub needs to provide enough power on booting up.

> power issue leading to corrupt "/" or any other SD card failure

For that, I have a backup of the sd card. I am using an "industrial" SD card by the way which costs a few bucks more. Also, the card is running in read-only mode, using unionfs. Logs are written to a partition of the RAID disks.

> usb disk breaking

That's handled by RAID1 and backups.

> OS crash borking everything

I've never seen that on Linux, in 22 years using it. The closest I came to was botched EFI parameters from an Ubuntu install gone wrong, on a Desktop PC. Wait, I also had a problem in about 2003 with a broken ISA disk controller. Ah, and I am using EXT3 which is a journaling file system.

> Raid might protect against a usbdisk breaking

Yeah, that is what it is for. It is not a substitute for backups (but nobody said something like that, I think). In fact, it is easier to make backups from a Pi, using the standard Linux tools like tar, ssh and rsync.

> its not even going to give you a performance boost either, its USB3

For my purposes, it does not need to be faster (and it is much, much faster than the old DNS-232). The Pi 400 has Gigabit Ethernet and plenty of RAM. I am running a MoinMoin wiki instance on it, and it is more than fast enough for personal information management.

It can well be that a small thing like that is not fast enough for a medium-scale business with, say, a dozen clients. But this was not what my requirements were. I wanted to have a solution which is very low-power. And the Pi 400 is probably one or two Watt if idle, and the disks are spun down:

https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications

https://www.raspberrypi-spy.co.uk/2018/11/raspberry-pi-power...

> This is why n+1 is better

So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Wednesday April 14, 2021 @12:17PM (#26809193)
Just to complete this:

As said, it is not a good idea to leave the root file system of the Raspberry, which is on an SD card, writable for heavy-duty, long-term tasks where reliability is important. It is not made for that. Among other things, this includes logging.

Instead, I am using this script:

https://github.com/fitu996/overlayRoot.sh

which mounts an overlay-filesystem with the bottom layer being the root-fs, the upper layer a new fs on writable media, which is a RAID partition in my case, and swaps the overlayfs with the root fs. That means that from then, the sd card of the Raspberry is not modified, and all changes go to the RAID disk. If you want to modify the system configuration, you need to re-boot with a different kernel parameter, which is detailed on the linked page.

Re:The latest DSM update makes btrfs drives unavailable on b... by KaiserPro on Thursday April 15, 2021 @06:19AM (#26818546)
> > OS crash borking everything

> I've never seen that on Linux, in 22 years using it

Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition.

> So, what kind of set-up are you exactly talking about, a distributed system of some kind? A storage cluster? Ceph? DynamoDB?

Noooooope!

just a second machine, with a copy of whatever you want to keep. when you are doing backups, have each one as a round robin target.

Ceph is not something you'd want to run for redundancy/reliability. I've spent many years supporting "HA" filesystems. The simpler the better (unless its GPFS, that's actually good.)

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Thursday April 15, 2021 @06:43AM (#26818734)
> Its surprisingly common, especially when a machine is underload and spending ages flushing to disk. A kernel oops happens and bam, broken partition.

1. That is what journaling file systems like ext4 are for. These are exactly to prevent that unfinished writes of metadata corrupt the file system.

2. Of course, it is possible that the Linux kernel goes all bonkers, writing random stuff to you disk and firing up nuclear missiles. But I have never seen that. I also have never seen a kernel Oops on a NAS. Kernel crashes are usually caused by faulty device drivers.

3. Also, in this aspect, there is no technical difference at all to a commercial NAS since they use Linux as well. They also use ARM CPUs, like the Raspberry, since they save power and do not need a fan (which in turns makes them less prone to failures).

> just a second machine, with a copy of whatever you want to keep.

Keeping backups is something I would seriously recomment for any kind of NAS or server.

I do really not see how your argument applies in any way to a NAS based on a Raspberry Pi. The main difference I see is that it is substantially cheaper, and better suited to people who want to install extra software and occasionally like to tinker a bit. And I am obviously not recommending that the Central Bank of Canada or an institution like that uses it.

Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Thursday April 15, 2021 @05:22AM (#26818126)
> The issue is not speed, its the USB disks. firstly you are reliant on the disks presenting themselves in the same order at the same time. (historically that was challenging) ZFS can cope with this quite well, MDADM, not so much.

I have checked about that and mdadm uses uuids to identify partitions on USB disks:

https://serverfault.com/questions/739580/mdadm-can-i-reconne...

https://serverfault.com/questions/460138/mdadm-disk-configur...

https://www.linuxquestions.org/questions/linux-software-2/md...

Here is how it looks on my array:

   root@arau:~# mdadm --detail /dev/md1
   /dev/md1:
              Version : 1.2
        Creation Time : Thu Dec 10 08:56:22 2020
           Raid Level : raid1
           Array Size : 4821105600 (4597.76 GiB 4936.81 GB)
        Used Dev Size : 4821105600 (4597.76 GiB 4936.81 GB)
         Raid Devices : 2
        Total Devices : 2
          Persistence : Superblock is persistent
   
        Intent Bitmap : Internal
   
          Update Time : Thu Apr 15 11:18:54 2021
                State : clean 
       Active Devices : 2
      Working Devices : 2
       Failed Devices : 0
        Spare Devices : 0
   
   Consistency Policy : bitmap
   
                 Name : arau2:1
                 UUID : 7c1e57ea:1f7ebff2:415c1b0a:8f959031
               Events : 21463
   
       Number   Major   Minor   RaidDevice State
          0       8        2        0      active sync      /dev/sda2
          1       8       18        1      active sync     /dev/sdb2
   
   root@arau:~#

   root@arau:~# mdadm --examine /dev/sda2
   /dev/sda2:
             Magic : a92b4efc
           Version : 1.2
       Feature Map : 0x1
        Array UUID : 7c1e57ea:1f7ebff2:415c1b0a:8f959031
              Name : arau2:1
        Creation Time : Thu Dec 10 08:56:22 2020
        Raid Level : raid1
      Raid Devices : 2
   
    Avail Dev Size : 9642249512 (4597.78 GiB 4936.83 GB)
        Array Size : 4821105600 (4597.76 GiB 4936.81 GB)
     Used Dev Size : 9642211200 (4597.76 GiB 4936.81 GB)
       Data Offset : 264192 sectors
      Super Offset : 8 sectors
      Unused Space : before=264112 sectors, after=38312 sectors
             State : clean
       Device UUID : 52594d2c:77f9c0cc:0b4243f6:6a8e6bcb
   
   Internal Bitmap : 8 sectors from superblock
       Update Time : Thu Apr 15 11:18:54 2021
     Bad Block Log : 512 entries available at offset 32 sectors
          Checksum : ca81c1ad - correct
            Events : 21463
   
   
      Device Role : Active device 0
      Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
   root@arau:~# 
You see the second UUID entry? It is an USB disk.

And I looked into the madm changelog and git log and I did not find any reference to problems as you describe since 2014. I am a bit confused why you refer to hypothetical problems or ones which do not exist anymore for years?

Re:The latest DSM update makes btrfs drives unavailable on b... by sokoloff on Wednesday April 14, 2021 @07:11AM (#26805479)
Clean shutdowns are indeed preferred, but synology devices will do a clean shutdown (delayed as required) if you press and hold the power button until it beeps and starts flashing, then release.
Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Wednesday April 14, 2021 @08:55AM (#26806382)
Yeah. That is a problem with older Raspberry Pis, which do not have a keyboard. It is possible to use a GPIO port for this, and shut it down with an attached button. But one has to be careful not to use the port unless it is configured properly, because that could cause a short-circuit, and the R-Pi is not protected against that. Having a keyboard on which you can type Ctrl-Alt-Del is much easier to use and will work on most relevant circumstances. In modern Raspian editions, this can be configured in systemd.
Re:The latest DSM update makes btrfs drives unavailable on b... by jnxx on Wednesday April 14, 2021 @09:02AM (#26806470)
I'd be really curious to know why people are downvoting this. I can't see any technical reason against it. Of course, for some people this might be too complicated or too much hassle in comparison to buying a more expensive commercial thing. But for other people, the cost savings might be worth the work. It is also easier to keep an almost completely open source solution fully up-to-date.