News for hackers. Stuff that matters.
Re:Pack: A New Container Format for Compressed Files by TylerE on Friday March 22, 2024 @04:34PM (#39794534)
Or just what zip and every other format does an skits put all the metadata at the beginning - enough to list all files, and extract any single one efficiently
Re:Pack: A New Container Format for Compressed Files by monocasa on Friday March 22, 2024 @04:51PM (#39794731)
zip interestingly sticks the metadata at the end. That lets you add files to a zip without touching what's already been zipped. Just new metadata at the end.

Modern tape archives like LTFS do the same thing as well.

Re:Pack: A New Container Format for Compressed Files by t43562 on Saturday March 23, 2024 @02:01AM (#39797803)
That sounds like you need to have fetched the whole zip before you can unzip it - which is not what one wants when making "virtual tarfiles" which only exist in a pipe. (i.e. you're packing files in at one end of the pipe and unpacking them at the other)
Re:Pack: A New Container Format for Compressed Files by bdhcuidbebe on Saturday March 23, 2024 @04:49AM (#39798446)
Just fseek to the end.

Zip format was not designed for piping.

Re:Pack: A New Container Format for Compressed Files by nullindividual on Friday March 22, 2024 @04:38PM (#39794586)
Tapes don't (? certainly didn't) operate this way. You need to read the entire tape to list the contents.

Since tar is a Tape ARchive, the way tar operates makes sense (as it was designed for both output to file and device, i.e. tape).

Re:Pack: A New Container Format for Compressed Files by monocasa on Friday March 22, 2024 @04:50PM (#39794718)
Tapes currently don't really operate like tar anymore either. Filesystems like LTFS stick the metadata all in one blob somewhere.
Re:Pack: A New Container Format for Compressed Files by nullindividual on Friday March 22, 2024 @05:00PM (#39794808)
It's been a long time since I've operated tape, so good to know things have changed for the better.
Re:Pack: A New Container Format for Compressed Files by tredre3 on Friday March 22, 2024 @04:58PM (#39794800)
That point is always raised on every criticism of tar (that it's good at tape).

Yes! It is! But it's awful at archive files, which is what it's used for nowadays and what's being discussed right now.

Over the past 50 years some people did try to improve tar. People did develop ways to append a file table at the end of an archive file. Maintaining compatibility with tapes, all tar utilities, and piping.

Similarly, driven people did extend (pk)zip to cover all the unix-y needs. In fact the current zip utility still supports permissions and symlinks to this day.

But despite those better methods, people keep pushing og tar. Because it's good at tape archival. Sigh.