News for hackers. Stuff that matters.

A comment on Pypi.org is running a survey on the state of Python packaging. Parent comment

Re:Pypi.org is running a survey on the state of Python packa... by bno1 on Wednesday September 07, 2022 @06:16PM (#32757780)
I wish pip had some package deduplication implemented. Even some basic local environments have >100MB of dependencies. ML environments go into the gigabytes range from what I remember.
[ Parent | on HN ]
Re:Pypi.org is running a survey on the state of Python packa... by smeagull on Wednesday September 07, 2022 @07:08PM (#32758425)
Cargo allows you to share a single package directory. Having a single site-packages and the ability to import a particular version (or latest by default) would solve this.
[ Parent | on HN ]
Re:Pypi.org is running a survey on the state of Python packa... by Asraelite on Wednesday September 07, 2022 @07:57PM (#32759023)
This is also nice because you can clean up your build files all in one go. Every now and again I delete my cargo target directory and get 5GB back.
[ Parent | on HN ]
Re:Pypi.org is running a survey on the state of Python packa... by BerislavLopac on Thursday September 08, 2022 @03:32AM (#32762450)
I'm not sure what you mean - unlike npm, pip installs only one copy of a dependency in each environment.
[ Parent | on HN ]
Re:Pypi.org is running a survey on the state of Python packa... by duckmysick on Thursday September 08, 2022 @05:00AM (#32763074)
So if I have environments A, B, and C and each has a dependency ML-package-1.2 that's 100 MB, it means there's one copy of it in each environment? Meaning 3x100 MB?
[ Parent | on HN ]
Re:Pypi.org is running a survey on the state of Python packa... by dagw on Thursday September 08, 2022 @05:08AM (#32763123)
Meaning 3x100 MB?

By default, yes. It is possible to install ML-package-1.2 into your 'base' python and then have virtual environments A,B and C all use that instead of installing their own copies. However this is generally not considered best practice.

[ Parent | on HN ]
Re:Pypi.org is running a survey on the state of Python packa... by BerislavLopac on Thursday September 08, 2022 @07:48AM (#32764191)
Ah, so across multiple environments? You do have a point, but that has nothing to do with pip - it's not an environment manager.

I can see that as an option in tools like venv or poetry, possibly setting a "common" location to share across environments.

[ Parent | on HN ]