Re:Show HN: Compressed directories on Linux
by Filligree on Sunday May 07, 2017 @07:12PM
(#14288217)
Or you could use a base filesystem that supports compression, such as ZFS. shrug
[ Parent
| on HN ]
What does the calculation for file system compression look like these days? On one hand we've got faster SSDs, but on the other we've got more spare cores and SSD space is relatively expensive compared to HDD space.
With LZ4 overhead is rather low, generally you'd get higher end to end throughput with LZ4 enabled. It's not as space efficient as more expensive algorithms but great for high performance scenarios.
But if you just want one directory on an existing filesystem to be compressed, this might be an easier option
The article's suggestion means eating up 50G of allocation straight away. I don't think needing to estimate your compressed allocation up front is an easier solution at all. Normally with directories you use thin provisioning pulling from the whole device's free space; thick provisioning for a directory is not a path for easier reduce space consumption.
Re:Show HN: Compressed directories on Linux
by givemefive on Sunday May 07, 2017 @08:33PM
(#14288492)
On top of each vdev (tank) you can make datasets (tank/logs) and mount those wherever
Each of these datasets can have their own options set. So yeah it's basically the same thing as btrfs?
Re:Show HN: Compressed directories on Linux
by funkaster on Sunday May 07, 2017 @09:18PM
(#14288635)
You could specify a dataset that mounts to a specific (sub)directory. Or even have a file-backed pool just like the example.
you could also do the trick with zfs - just create an zpool on that loop device - this also gives you lz4 and gzip
Yes, but ZFS is probably more memory hungry and somewhat an overkill just for compression alone. It's a pity simpler filesystems don't have transparent compression.
Re:Show HN: Compressed directories on Linux
by matt_wulfeck on Monday May 08, 2017 @01:50AM
(#14289517)
I'm not sure it's an overkill compared to the presented solution.
Re:Show HN: Compressed directories on Linux
by keeperofdakeys on Monday May 08, 2017 @06:12AM
(#14290429)
One disadvantage of zfs in this case is that it uses its own caching layer, since it's basically an OS subsystem in itself. So on a memory constrained host (eg. rpi), you might have some issues with the zfs and linux buffers competing for RAM.
Re:Show HN: Compressed directories on Linux
by trentnelson on Monday May 08, 2017 @04:43PM
(#14295407)
Like NTFS has had for the last 20+ years? ;-)
One of the benefits of NT's packet-driven I/O model (versus the UNIX synchronous VFS read/write blocking buffer I/O model).