News for hackers. Stuff that matters.
Re:Show HN: Compressed directories on Linux by Filligree on Sunday May 07, 2017 @07:12PM (#14288217)
Or you could use a base filesystem that supports compression, such as ZFS. shrug
Re:Show HN: Compressed directories on Linux by flukus on Monday May 08, 2017 @02:56AM (#14289760)
What does the calculation for file system compression look like these days? On one hand we've got faster SSDs, but on the other we've got more spare cores and SSD space is relatively expensive compared to HDD space.
Re:Show HN: Compressed directories on Linux by hultner on Monday May 08, 2017 @06:11AM (#14290422)
With LZ4 overhead is rather low, generally you'd get higher end to end throughput with LZ4 enabled. It's not as space efficient as more expensive algorithms but great for high performance scenarios.
Re:Show HN: Compressed directories on Linux by polygot on Sunday May 07, 2017 @07:55PM (#14288363)
But if you just want one directory on an existing filesystem to be compressed, this might be an easier option
Re:Show HN: Compressed directories on Linux by barrkel on Monday May 08, 2017 @05:03AM (#14290175)
The article's suggestion means eating up 50G of allocation straight away. I don't think needing to estimate your compressed allocation up front is an easier solution at all. Normally with directories you use thin provisioning pulling from the whole device's free space; thick provisioning for a directory is not a path for easier reduce space consumption.
Re:Show HN: Compressed directories on Linux by givemefive on Sunday May 07, 2017 @08:33PM (#14288492)
On top of each vdev (tank) you can make datasets (tank/logs) and mount those wherever

Each of these datasets can have their own options set. So yeah it's basically the same thing as btrfs?

Re:Show HN: Compressed directories on Linux by funkaster on Sunday May 07, 2017 @09:18PM (#14288635)
You could specify a dataset that mounts to a specific (sub)directory. Or even have a file-backed pool just like the example.
Re:Show HN: Compressed directories on Linux by nisa on Monday May 08, 2017 @12:12AM (#14289191)
you could also do the trick with zfs - just create an zpool on that loop device - this also gives you lz4 and gzip
Re:Show HN: Compressed directories on Linux by shmerl on Monday May 08, 2017 @12:14AM (#14289199)
Yes, but ZFS is probably more memory hungry and somewhat an overkill just for compression alone. It's a pity simpler filesystems don't have transparent compression.
Re:Show HN: Compressed directories on Linux by matt_wulfeck on Monday May 08, 2017 @01:50AM (#14289517)
I'm not sure it's an overkill compared to the presented solution.
Re:Show HN: Compressed directories on Linux by keeperofdakeys on Monday May 08, 2017 @06:12AM (#14290429)
One disadvantage of zfs in this case is that it uses its own caching layer, since it's basically an OS subsystem in itself. So on a memory constrained host (eg. rpi), you might have some issues with the zfs and linux buffers competing for RAM.
Re:Show HN: Compressed directories on Linux by trentnelson on Monday May 08, 2017 @04:43PM (#14295407)
Like NTFS has had for the last 20+ years? ;-)

One of the benefits of NT's packet-driven I/O model (versus the UNIX synchronous VFS read/write blocking buffer I/O model).

Re:Show HN: Compressed directories on Linux by shmerl on Monday May 08, 2017 @04:58PM (#14295533)
BTRFS is now good for it, but having it in other filesystems would be useful too. Linux now supports some async I/O as well.