News for hackers. Stuff that matters.
Re:ZFS: Use mirror vdevs, not RAIDZ by aidenn0 on Wednesday December 09, 2020 @12:00PM (#25360754)
I was running 3x 12 drive vdevs in raidz2 and write performance was terrible. We moved some data to a different machine and rebuilt as 18 mirrors. This was a long time ago (as in running on Solaris long time ago), so maybe things are better now.
Re:ZFS: Use mirror vdevs, not RAIDZ by segfaultbuserr on Wednesday December 09, 2020 @04:47PM (#25365677)
Slow performance is always my impression with RAIDZ. There's an obvious performance hit if you are coming from conventional RAID setups (like mdadm). I've even seen some unbelievable slow speed like 1 MiB/s in the middle of copying a large git repo (which have a lot of small files, repeated git pull also creates a lot of fragmentation). But my experience was based on early ZFSOnLinux, perhaps a real BSD will perform better. Anyway, I knew ZFS's first goal is data safety, not performance, and it's why I could tolerate the performance...
Re:ZFS: Use mirror vdevs, not RAIDZ by myrond on Wednesday December 09, 2020 @05:13PM (#25366001)
Each 12 drive array would have an interesting physical block layout.

What was your ASHIFT and what was your physical disk raw sector size because if you had a ASHIFT=9 you would have 5120k blocks, if it was a ASHIFT=12 you would have physical 40960 blocks. None of those line up well with zfs.

Additionally if this was a long time ago the pool would have defaulted to ASHIFT=9 and if you added any 4K drives you could have been having write amplification occurring.

Looks as if you had your drives doing a lot of unnecessary extra wasted work.

Re:ZFS: Use mirror vdevs, not RAIDZ by fomine3 on Wednesday December 09, 2020 @11:10PM (#25369601)
This is huge thing (called "Slop Space") that I noticed after I bought all ZFS NAS parts!

It wastes much capacity by default if drive count is not ideal. IIRC use larger recordsize (1MB or above) and enable compression almost solves issue. I use default 128K recordsize for small files zvol for performance, and use larger recordsize for the rest of zvols.

https://wintelguy.com/zfs-calc.pl

Re:ZFS: Use mirror vdevs, not RAIDZ by aidenn0 on Wednesday December 09, 2020 @08:33PM (#25368392)
Ashift=9 native 512b sector drives
Re:ZFS: Use mirror vdevs, not RAIDZ by seized on Thursday December 10, 2020 @10:12AM (#25374477)
RAIDZ gives roughly the IOPS of one drive per VDEV. Mirrors is one drive of write IOPS per VDEV and some say two drives of IOPS for read.

You went from 3 drives to 18, IOPS wise.