News for hackers. Stuff that matters.
Re:A better zip bomb by hinkley on Thursday July 04, 2019 @02:07PM (#20356564)
See, I don’t understand this. Zip is a streamable format. I don’t understand why you would extract the archive before checking the contents?

I worked on productizing a code signing tool a while back and I believe the first thing I did after we got it working was change it so nothing touched the disk until after the signature had been validated (in this case the signers had business relationships with each other. This would be necessary but insufficient for download s from the internet).

There were already well known CERT advisories about how relative paths can confuse archive tools, email tools and web servers from Microsoft. Know history or repeat it.

I didn’t know “zip bomb” as a phrase but I knew a good bit about compression, so when I needed to fix a problem with zips over 2G I managed to make myself a test fixture that was around 80k without modifying the file format. I think it was just 2.01G of white space.

Re:A better zip bomb by deathanatos on Thursday July 04, 2019 @06:04PM (#20358170)
> Zip is a streamable format

Not really. For a normal, created in one-shot ZIP, yes. But the types of ZIPs in the article are going to act differently if you tried streaming them. The core idea of the article is overlapping the various files within the ZIP, s.t. they share bytes. But this is only apparent if you're using the central directory, which you can't if you're streaming, since it appears after all the data. If you're streaming, you're using the local file headers, but for ZIPs such as those in the article, you will see many less LHFs than had you looked them up in the central directory, b/c they overlap. (In the streaming case, I think you'd see exactly 1 file. It would still be huge, given the other tricks in the article.)

Also, I think you can "append" to ZIPs (this is why the central directory is at the end, s.t. it can be overwritten by new data, and then re-appended.) I think this approach allows tools to also "delete" data by simply removing the entry from the central directory, and re-appending it w/o, so the central directory is essentially the authoritative source for the ZIPs contents. (Though I suppose a streaming decompressor could decompress to a temporary location and then only move non-deleted entries into their final place.)

The Wikipedia page echos this:

> Tools that correctly read ZIP archives must scan for the end of central directory record signature, and then, as appropriate, the other, indicated, central directory records. They must not scan for entries from the top of the ZIP file, because (as previously mentioned in this section) only the central directory specifies where a file chunk starts and that it has not been deleted. Scanning could lead to false positives, as the format does not forbid other data to be between chunks, nor file data streams from containing such signatures.

https://en.wikipedia.org/wiki/Zip_(file_format)

Re:A better zip bomb by hinkley on Thursday July 04, 2019 @09:24PM (#20359233)
Why is everybody fixating on my preface?

> I don’t understand why you would extract the archive before checking the contents?

Re:A better zip bomb by girvo on Thursday July 04, 2019 @11:26PM (#20359593)
Because a curious part of written internet culture is that of fixating on small mistakes that aren't even important to the overall context or meaning of a post, and ignoring the actual content. Why, I'm unsure, but I've been noticing it a lot more now that I'm aware of it!
Re:A better zip bomb by hinkley on Friday July 05, 2019 @05:16AM (#20361020)
Been thinking the same, but it also often feels like “waiting for your turn to talk”.
Re:A better zip bomb by charonn0 on Thursday July 04, 2019 @06:47PM (#20358405)
> See, I don’t understand this. Zip is a streamable format.

Zip is a streamable format but it also supports random access through a table of contents (the "central directory") located at the end of the file. This bomb works by overlapping the file offsets in the table of contents.

Re:A better zip bomb by hinkley on Thursday July 04, 2019 @09:24PM (#20359225)
> I don’t understand why you would extract the archive before checking the contents.
Re:A better zip bomb by charonn0 on Friday July 05, 2019 @12:27AM (#20359838)
If by "checking" you mean to examine the archive structure to determine whether it's corrupt, and "extracting" you mean "write the archive's contents to disk" then they are fundamentally the same thing. The only difference is that checking sends the content to /dev/null instead of a file.

As to why they're writing the contents to disk I can only speculate. Perhaps they're using a library that doesn't expose an "extract to memory" feature, or maybe it's an anti-zipbomb measure to avoid out-memory/denial-of-service attacks.

Re:A better zip bomb by hinkley on Friday July 05, 2019 @05:13AM (#20361005)
Have you ever done any work in this area? Because it sounds like you know what you’re talking about, except it’s all nonsense.

Zip format can be de/compressed progressively, which is one reason why it’s nice for HTTP transport encoding. The file format is decompressed one record at a time and many or most libraries can give you this as a stream, so it never has to hit disk or be “sent to dev/null”.

If you take responsibility for streaming the records to disk (trivial), then you can check the canonical path before writing, and any other filesystem sanity tests you want to do.

Re:A better zip bomb by charonn0 on Friday July 05, 2019 @02:45PM (#20364778)
Last year I implemented zip reading and zip writing in a hobby project of mine. I'm not an expert, but I know enough to write a working zip reader/writer.

> Zip format can be de/compressed progressively, which is one reason why it’s nice for HTTP transport encoding.

Do you mean HTTP transfer encoding? If so then it's not the zip archive format that's used, but rather the deflate compression algorithm (which zip also uses.)

> The file format is decompressed one record at a time

But not necessarily in the order they appear.

> many or most libraries can give you this as a stream, so it never has to hit disk or be “sent to dev/null”.

My point is that the compressed bytes have to be decompressed and checksummed in both extraction and checking, but after that the bytes may either be written or discarded.

> If you take responsibility for streaming the records to disk (trivial), then you can check the canonical path before writing, and any other filesystem sanity tests you want to do.

That's true but there's nothing wrong with the paths in this case.