News for hackers. Stuff that matters.
Re:Uv's killer feature is pulling in local dependencies by mdaniel on Monday January 13, 2025 @10:56AM (#42684819)
I wish the ecosystem made heavier use of .zip packages, which would gravely help with the logistics of your hardlink plan, in addition to slimming down 300MB worth of source code. The bad news is that (AIUI) the code needs to be prepared for use from a package and thus my retroactively just zipping them up will break things at runtime

Take for example:

  $ du -hs $HOMEBREW_PREFIX/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/* | gsort --human
  103M /usr/local/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/botocore
  226M /usr/local/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/ansible_collections
Re:Uv's killer feature is pulling in local dependencies by zahlman on Monday January 13, 2025 @11:52AM (#42685508)
Wheels are zip files with a different extension. And the Python runtime can import Python code from them - that's crucial to the Pip bootstrapping process, in fact.

But packages that stay zipped aren't a thing any more - because it's the installer that gets to choose how to install the package, and Pip just doesn't do that. Pip unpacks the wheel mainly because many packages are written with the assumption that the code will be unpacked. For example, you can't `open` a relative path to a data file that you include with your code, if the code remains in a zip file. Back in the day when eggs were still a common distribution format, it used to be common to set metadata to say that it's safe to leave the package zipped up. But it turned out to be rather difficult to actually be sure that was true.

The wheel is also allowed to contain a folder with files that are supposed to go into other specified directories (specific ones described by the `sysconfig` standard library https://docs.python.org/3/library/sysconfig.html ), and Pip may also read some "entry point" metadata and create executable wrappers, and leave additional metadata to say things like "this package was installed with Pip". The full installation process for wheels as designed is described at https://packaging.python.org/en/latest/specifications/binary... . (Especially see the #is-it-possible-to-import-python-code-directly-from-a-wheel-file section at the end.)

The unzipped package doesn't just take up extra space due to unzipping, but also because of the .pyc cache files. A recent Pip wheel is 1.8 MiB packed and 15.2 MiB unpacked on my system; that's about 5.8 apparent MiB .py, 6.5 MiB .pyc, 2.1 MiB from wasted space in 4KiB disk blocks, and .8 MiB from a couple hundred folders.