Re:Ask HN: Why does validating a user require 14000 files?
by sieve on Monday August 30, 2021 @08:10AM
(#28354832)
This insanity is far more pervasive and has infected software distribution as well.
A couple of years back I installed a copy of Sonic Pi[1] on my machine and was horrified to see that the distribution basically dumped tens of thousands of extremely tiny Ruby files on the hard disk. Ever tried copying hundreds of thousands of sub 1 KB files from one disk to another and noticed the file system crying under the load?
Game developers solved this problem decades ago.[2][3] I don't know why other developers continue to be so backward in their thinking when distributing their software. You don't have to do anything special. Just use SQLite as a VFS and end the insanity.[4]
A couple of years back I installed a copy of Sonic Pi[1] on my machine and was horrified to see that the distribution basically dumped tens of thousands of extremely tiny Ruby files on the hard disk. Ever tried copying hundreds of thousands of sub 1 KB files from one disk to another and noticed the file system crying under the load?
Game developers solved this problem decades ago.[2][3] I don't know why other developers continue to be so backward in their thinking when distributing their software. You don't have to do anything special. Just use SQLite as a VFS and end the insanity.[4]
[1] https://github.com/sonic-pi-net/sonic-pi
[2] https://quakewiki.org/wiki/.pak
[3] https://wiki.totalwar.com/w/Community_hints,_tips_and_tutori...
[4] https://sqlite.org/appfileformat.html