News for hackers. Stuff that matters.
Re:Uv: Python packaging in Rust by eigenvalue on Thursday February 15, 2024 @03:13PM (#39388080)
Looks awesome. I find that pip is usually pretty fast for me, and when it's not, it is mostly because it has to download so much data or wait for native libraries to compile in the background (or anything involving cuda which always seems to take forever). What really needs some help with speed is conda, which is just so absurdly slow for literally anything, even on ridiculously powerful machines.
Re:Uv: Python packaging in Rust by rogue7 on Thursday February 15, 2024 @03:36PM (#39388406)
For conda there is mamba [0], a drop-in replacement that's really fast.

By the way, the creator of mamba started his own company at https://prefix.dev/

They want to essentially leverage the conda(-forge) infrastructure to build a new cross-platform, cross-language, cargo-like package manager: pixi

[0] https://github.com/mamba-org/mamba

Re:Uv: Python packaging in Rust by maxnoe on Thursday February 15, 2024 @04:07PM (#39388879)
Since last year, conda itself is actually using the mamba solver:

https://conda.org/blog/2023-11-06-conda-23-10-0-release/

Re:Uv: Python packaging in Rust by daniel_grady on Thursday February 15, 2024 @04:25PM (#39389139)
What are some of the reasons that teams use conda (and related tools) today? As a machine learning scientist, I used conda exclusively in the mid-2010s because it was the only framework that could reliably manage Python libraries like NumPy, PyTorch, and so on, that have complex binary dependencies. Today, though, pip install works fine for those packages. What am I missing?
Re:Uv: Python packaging in Rust by blactuary on Thursday February 15, 2024 @08:12PM (#39391763)
For me personally, I prefer conda because it is dependency resolution (mamba), virtual environments, and a package repository (conda-forge) all from one base miniconda installation. And for all of my use cases, all of those just work. Dependency solving used to be painfully slow, mamba solved that. Packages used to be way behind the latest, setting conda-forge as my default solved that.

After fiddling with different solutions for years and having to start fresh with a new Python install, I've been using nothing by miniconda for years and it just works

Re:Uv: Python packaging in Rust by Ringz on Thursday February 15, 2024 @05:16PM (#39389922)
Unfortunately, far too often: tradition.

Using only „Pythons native tools“ like pip and venv simply works nowadays so good that I wonder about the purpose of many tools like poetry etc. etc.

Re:Uv: Python packaging in Rust by gcarvalho on Thursday February 15, 2024 @08:12PM (#39391770)
For me it's the easiest and fastest cross-platform way to consistently install a Python version.

pip and venv work fine, but you have to get them first; and that can be a struggle for unseasoned python devs, especially if you need a version that's not what your distro ships, and even more so on Windows and macOS.

I use micromamba [1] specifically, which is a single binary.

[1] https://mamba.readthedocs.io/en/latest/user_guide/micromamba...

Re:Uv: Python packaging in Rust by RockRobotRock on Thursday February 22, 2024 @08:00AM (#39466415)
Maybe it's because I came into Python later, but I've almost never had the problem of pip not being installed. That's what ensurepip is for, right?
Re:Uv: Python packaging in Rust by pininja on Thursday February 15, 2024 @05:32PM (#39390151)
Another reason I used to use conda was for easy native Windows installation. GPU accelerated packages like OpenCV were especially difficult when I used use it 6 years ago. Now there’s Linux subsystem.. has pip support dramatically improved on Windows?
Re:Uv: Python packaging in Rust by nateglims on Thursday February 15, 2024 @05:42PM (#39390260)
The biggest advantage for poetry I found, working with a lot of non-traditional software people, is that it does a lot of things by default like pin versions and manage virtual envs. Unfortunately, it does complicate some things.
Re:Uv: Python packaging in Rust by Ringz on Thursday February 15, 2024 @05:51PM (#39390371)
I can understand that well. A few articles from ByteCode! helped me to "follow my intuition" and do as much as possible with native Python tools.

https://www.bitecode.dev/p/back-to-basics-with-pip-and-venv

https://www.bitecode.dev/p/relieving-your-python-packaging-p...

Re:Uv: Python packaging in Rust by daniel_grady on Thursday February 15, 2024 @07:18PM (#39391268)
Those are interesting pointers; appreciate it! My own experience over the past three years has been similar. I tried using Pipenv, and then Poetry, for internal projects at my company; in both cases the tool seemed overly complicated for the problem, slow, and I had a hard time getting co-workers on board. About a year and a half ago, I saw [Boring Python: dependency management](https://www.b-list.org/weblog/2022/may/13/boring-python-depe...), which recommends using the third-party `pip-tools` library alongside the standard library’s `pip` and `venv`, and switched to that for the next project. It’s been working great. The project has involved a small team of scientists (four or five, depending) who use a mix of macOS and Windows. We do analysis and development locally and write production-facing algorithms in Python packages tracked in our repository, and publish releases to Gitlab’s PyPI. For our team, the “get up and running” instructions are “clone, create a venv, and pip install -r requirements.txt” and for the software team that manages the production systems, deploying an update just means pip installing a new version of the package. Every team’s got different constraints, of course, but this has been working very smoothly for us for over a year now, and it’s been easy, no pushback, with everyone understanding what’s going on. Really impressed with the progress of the core Python packaging infrastructure over the past several years.
Re:Uv: Python packaging in Rust by RockRobotRock on Thursday February 15, 2024 @07:00PM (#39391089)
Has anyone else been paying attention to how hilariously hard it is to package PyTorch in poetry?

https://github.com/python-poetry/poetry/issues/6409

Re:Uv: Python packaging in Rust by dragonwriter on Thursday February 15, 2024 @06:12PM (#39390599)
> Today, though, pip install works fine for those packages.

pip install works, but pip's dependency management doesn't seem to (for Pytorch, specifically) which is why projects that have pip + requirements.txt as one of their installation methods will often have separate pytorch installation instructions when using that method, though if the same project supports conda installation it will be a one-stop-shop installation that way.

Re:Uv: Python packaging in Rust by daniel_grady on Thursday February 15, 2024 @06:43PM (#39390919)
> pip's dependency management doesn't seem to (for Pytorch, specifically)

That’s interesting — I’ve also had difficulties with PyTorch and dependency resolution, but only on the most recent versions of Python, for some period of time after they’re released. Picking Python 3.9 as a baseline for a project, for example, has been very reliable for PyTorch and all the related tooling.

Re:Uv: Python packaging in Rust by tehnub on Thursday February 15, 2024 @11:35PM (#39393196)
One reason to choose one over the other is the dependencies they’re bundled with. Take numpy. With PyPI, it’s bundled with OpenBLAS, and with conda, it’s bundled with Intel MKL, which can be faster. See https://numpy.org/install/#
Re:Uv: Python packaging in Rust by daniel_grady on Friday February 16, 2024 @02:53AM (#39394362)
That’s a great point; I didn’t know about that!
Re:Uv: Python packaging in Rust by paulddraper on Thursday February 15, 2024 @03:16PM (#39388128)
Yeah, I'm curious how much uv is actually faster.

npm -> Yarn was life changing performance-wise.

I wonder what pip -> uv is.

Re:Uv: Python packaging in Rust by the_mitsuhiko on Thursday February 15, 2024 @03:23PM (#39388240)
From my testing in rye it’s significantly faster in day to day. There are numbers in the blog post obviously but it’s the feeling you get using it locally that makes it much more fun to use.
Re:Uv: Python packaging in Rust by laborcontract on Thursday February 15, 2024 @03:24PM (#39388250)
Bun, ruff/uv, polars.. all have been major qol improvements.

I’m loving the pace of release with this crop of speed obsessed projects and I cannot wait for astral to tackle typing.

Re:Uv: Python packaging in Rust by laborcontract on Thursday February 15, 2024 @06:32PM (#39390804)
Ok, here are some benchmarks installing an application that has a couple build-heavy dependencies:

    Installation times (MM:SS):

    # No cache
    - uv:  01:05
    - pip: 01:56

    # Cache:
    - uv:  00:02
    - pip: 00:42
Re:Uv: Python packaging in Rust by eigenvalue on Thursday February 15, 2024 @03:39PM (#39388453)
npm seems to have gotten a lot faster lately. All that competition from yarn and now bun seems to have pushed them to focus on optimization.
Re:Uv: Python packaging in Rust by paulddraper on Thursday February 15, 2024 @03:47PM (#39388556)
Oh certainly.

Npm has improved greatly.

Npm features follow a two-step process: (1) Get implemented in Yarn (2) Get implemented in npm.

Re:Uv: Python packaging in Rust by driverdan on Thursday February 15, 2024 @10:35PM (#39392906)
I did some basic testing today and uv was around 2x faster than pip for a clean venv and cold cache on a real, decent sized project. With warm uv cache it was incredibly fast, under 10 sec.