I worked on productizing a code signing tool a while back and I believe the first thing I did after we got it working was change it so nothing touched the disk until after the signature had been validated (in this case the signers had business relationships with each other. This would be necessary but insufficient for download s from the internet).
There were already well known CERT advisories about how relative paths can confuse archive tools, email tools and web servers from Microsoft. Know history or repeat it.
I didn’t know “zip bomb” as a phrase but I knew a good bit about compression, so when I needed to fix a problem with zips over 2G I managed to make myself a test fixture that was around 80k without modifying the file format. I think it was just 2.01G of white space.
Not really. For a normal, created in one-shot ZIP, yes. But the types of ZIPs in the article are going to act differently if you tried streaming them. The core idea of the article is overlapping the various files within the ZIP, s.t. they share bytes. But this is only apparent if you're using the central directory, which you can't if you're streaming, since it appears after all the data. If you're streaming, you're using the local file headers, but for ZIPs such as those in the article, you will see many less LHFs than had you looked them up in the central directory, b/c they overlap. (In the streaming case, I think you'd see exactly 1 file. It would still be huge, given the other tricks in the article.)
Also, I think you can "append" to ZIPs (this is why the central directory is at the end, s.t. it can be overwritten by new data, and then re-appended.) I think this approach allows tools to also "delete" data by simply removing the entry from the central directory, and re-appending it w/o, so the central directory is essentially the authoritative source for the ZIPs contents. (Though I suppose a streaming decompressor could decompress to a temporary location and then only move non-deleted entries into their final place.)
The Wikipedia page echos this:
> Tools that correctly read ZIP archives must scan for the end of central directory record signature, and then, as appropriate, the other, indicated, central directory records. They must not scan for entries from the top of the ZIP file, because (as previously mentioned in this section) only the central directory specifies where a file chunk starts and that it has not been deleted. Scanning could lead to false positives, as the format does not forbid other data to be between chunks, nor file data streams from containing such signatures.
> I don’t understand why you would extract the archive before checking the contents?
Zip is a streamable format but it also supports random access through a table of contents (the "central directory") located at the end of the file. This bomb works by overlapping the file offsets in the table of contents.
As to why they're writing the contents to disk I can only speculate. Perhaps they're using a library that doesn't expose an "extract to memory" feature, or maybe it's an anti-zipbomb measure to avoid out-memory/denial-of-service attacks.
Zip format can be de/compressed progressively, which is one reason why it’s nice for HTTP transport encoding. The file format is decompressed one record at a time and many or most libraries can give you this as a stream, so it never has to hit disk or be “sent to dev/null”.
If you take responsibility for streaming the records to disk (trivial), then you can check the canonical path before writing, and any other filesystem sanity tests you want to do.
> Zip format can be de/compressed progressively, which is one reason why it’s nice for HTTP transport encoding.
Do you mean HTTP transfer encoding? If so then it's not the zip archive format that's used, but rather the deflate compression algorithm (which zip also uses.)
> The file format is decompressed one record at a time
But not necessarily in the order they appear.
> many or most libraries can give you this as a stream, so it never has to hit disk or be “sent to dev/null”.
My point is that the compressed bytes have to be decompressed and checksummed in both extraction and checking, but after that the bytes may either be written or discarded.
> If you take responsibility for streaming the records to disk (trivial), then you can check the canonical path before writing, and any other filesystem sanity tests you want to do.
That's true but there's nothing wrong with the paths in this case.
I'm afraid to even download it now...
I prefer this functionality as most of the time I do want to unarchive it. I can rearchive it later (or remember to use another browser) when I need to.
MS Windows' transparent zip treated as folders works well (it's the same on Kubuntu), one could just do that and pre-cache an uncompressed version; then you get to use the file or open the folder with minimal friction?
——
Also, fun bonus fact re: checksumming:
Apple uses the https://en.m.wikipedia.org/wiki/Xar_(archiver) file format for their own software downloads (e.g. App Store downloads; and developer-tools packages from the Apple developer website; and downloading Safari extensions, before those were rolled into the App Store; etc.). Despite Apple being seemingly the sole user of .xar, it’s not an Apple-specific format; rather, it’s developed by OpenDarwin. So you can use it too (for, at least, your macOS-targeted downloads), if you like.
A .xar file contains embedded checksums (both for the archival representations of each file, and for their extracted representations); when Safari auto-unpacks a .xar, the .xar unpacker (Archive Utility?) verifies those checksums as it does so. IIRC, if the verification fails, the extraction stops, what has been extracted so far is deleted, and the user is told the archive is broken and asked whether they want to keep it or move it to the Trash.
A neat thing about .xar extraction, is that it seemingly tags the extracted files with an xattr declaring that they’ve already been checksummed. Apple ships applications like “Install macOS Whatever.app” as a .xar containing an .app bundle containing several mountable .dmg files; normally those .dmg files would do their own checksumming when they mount, but since they came out of a .xar, they know they’ve already been checksummed recently, so they just skip the internal checksumming step. (I think this is one of the main reasons Apple chose to move to .xar; they wanted to be able to make the macOS Installer run faster, by having it not have to do any checksums of its support .dmg files during install.)
So that’s the deeper answer to your question: ultimately, Apple expects people who want archives with checksums, to use .xar or a format like .xar, that does checksumming during extraction.
Purely my hypothesis: it’s too high a risk that someone accidentally leaves a password or private info in their clipboard. People don’t expect a clipboard to persist, so you’d need to re-educate everyone to avoid this “bug”, just like browser history and incognito mode.
Apple is notorious for assimilating popular third party extensions. Screencapture, night shift, colour picker , why no stack based clipboard? It’s too useful to have been overlooked. Must have been a conscious decision.
The only reason I've Chrome around is for the growing number of sites that only work with Chrome. Apps made by Google, in particular, increasingly don't support macOS/Safari. Which I find infuriating, but that's another topic.
When a browser helpfully decompresses the archive you can no longer perform this check.
But if someone replaces the archive with a malicious file that decompress normally, he will also probably change the listed checksum on the download page....
Many linux distro isos are available from several different mirrors. Having a secure hash on the original site with the links to mirrors means I don't have to trust the mirror(s).
Another case where the archive hash is useful is when there's some public key crypto involved. I can have a public key from a publisher (gotten either out-of-band or in the past) and the hash can be signed so I can verify it. These schemes would mean that an attacker would at the least need to have compromised a site for an extended period of time (if I have history with the site, the first visit it doesn't do anything extra), or in the case of out-of-band key sharing, multiple communication methods might need to be compromised for an attack to succeed.
But yes, in the common case a hash next to a file link hosted on the same domain really doesn't do anything.
You can compare the checksum to the one on the author's site to ensure the mirror provider didn't alter the file.
'Monitoring'.
As for malware that would be unzipped when using an external zip file; first you would need to trigger the zip bomb on defender but not the external tool, and second defender will still scan the individual files getting unziped by that tool.
The other possibility I see harks back to algorithms - build a DAG from the archive's interfile dependencies and run an iterative deepening search through the structure against some heuristic checking for malicious design.
I've never gotten into anything security related (other than reading Schneier's blog) but the cat and mouse game is fascinating.
I wonder how Windows Defender would treat something that looked like a self-extracting archive? Perhaps the archive portion could be this zip bomb affair, but the executable portion had a small change in it to bypass that and do something else nefarious instead, eg hand execution control to a point later in the file.
Windows 10 then began doing the same thing for Windows Defender, but some sane limits aborted it after a few seconds.