News for hackers. Stuff that matters.
Re:The Hardest Tarball? by rmast on Tuesday November 10, 2026 @11:22AM (#50442525)
I’ve been working on an open source tool for creating SBOMs from binaries/file systems. Automated extraction of archives like zip/tar files present interesting challenges - for a while I’ve known about OS filesystem formatting gotchas like archives from a Linux-based system with symlinks or case colliding names not extracting correctly on NTFS (and HFS+ for the latter case).

But the ambiguity of this tar files… I guess the “correct” extraction for what I’m doing would require figuring out what the system that is processing the tar files used and copy that behavior, which is non-trivial if you don’t know what an embedded device is using behind the scenes.

Re:The Hardest Tarball? by stabbles on Tuesday November 10, 2026 @11:36AM (#50442720)
Astral blogged about their new tar parser for uv recently, see https://astral.sh/blog/tar-codec. I've included it in my work-in-progress test suite https://stoppels.ch/tar/ which shows it's by far the strictest tar library out there.

For your use case it would make sense to pick the strictest tar parser, such that you can error or warn on most ambiguities. In practice these ambiguities are rare.

Re:The Hardest Tarball? by rmast on Tuesday November 10, 2026 @01:03PM (#50443857)
A strict parser makes sense for flagging that there are issues, but that also can’t block extraction; it needs to be configurable enough that users can make a decision on how to handle the ambiguity.
Re:The Hardest Tarball? by woodruffw on Tuesday November 10, 2026 @01:12PM (#50443961)
There are many scenarios (such as fetching package distributions over the internet) where the user lacks sufficient information to make an informed and secure decision about how to safely parse and extract a malformed tar (or any other archive format).

(Another way of saying that is that we live in a post-Postel world[1].)

[1]: https://alexgaynor.net/2025/mar/25/postels-law-and-the-three...

Re:The Hardest Tarball? by rmast on Tuesday November 10, 2026 @03:11PM (#50445654)
That’s true, but also much more general than our SBOM generator is targeting. We’re typically looking at filesystems pulled from embedded systems, or their update files. Ultimately we want to accurately represent what the state of the file system in the device looks like, which can require reverse engineering the update process to determine how to handle ambiguous cases (“correct” is matching what the device firmware update does).