zanie

> FYI: The extraction of files is largely python code that doesn't release the GIL. (cf. the zipfile class from the python interpreter has large layers of abstraction with a massive overhead in python code).

TIL. I ran some benchmarks and confirmed this is the case for many small files as you'd see in wheels — the GIL is released in a meaningful way for larger files though. Thanks!

> This might not give the same result for uv though.

Yeah, I built a prototype and ran some benchmarks. It makes a big difference if the entire wheel is empty files but for any real world examples it's within noise of the baseline.

You might be interested in taking a look at the `uv download` sketch I started on last week https://github.com/astral-sh/uv-dev/pull/875

We didn't compare to pip at the time, but it saved a lot of absolute time for us as reported in the benchmarks from the pull request (https://github.com/astral-sh/uv/pull/789) it improved a boto3 case by 3x (30s to 10s) and our "standard" solve benchmark by 2x. It's plausible some of those gains have been reduced by other optimizations in our solver since then though.

But this was just one example optimization, we do other low-level things, like zero-copy deserialization from our cache. The point is not that we do specific things, but that we have more levers to pull to improve performance. It's great to see all the improvements happening in pip performance regardless :)

I haven't done the comparison but it shouldn't be so hard

- https://docs.astral.sh/uv/reference/environment/#uv_concurre...

- https://docs.astral.sh/uv/reference/environment/#uv_concurre...

I'm a bit confused. pdm hit 1.0 in 2021, 3 years before uv was released.

(I work on uv)

> The only advantage of uv is to have support for parallel async extraction.

This isn't true, the biggest speed ups are for the warm cases where we've already unpacked the files into the cache, as notatallshaw mentions above. We can also make low-level optimizations during resolution, e.g., in version parsing, that are not possible in pure Python code.

> If pip extracted multiple files/wheels in parallel without being blocked by the GIL, pip could easily match or outcompete uv.

I'm a bit confused by these claims about the GIL? The expensive IO operations release the GIL.

> When the user edits one file, all files are modified simultaneously

This is why we default to reflinks or copy-on-write semantics when creating environments, not all file systems support it but it's becoming more common.

> Hypothetically, a simple deduplication of binary files (.dll .so) should achieve 50% of the savings without significant drawbacks

We also explored this (see https://github.com/astral-sh/uv/pull/19694) and the linked pull request has a table comparing to this strategy.

> We have optimization for empty files (0 bytes) because there is nothing to write and checksum.

Interesting, I would be very surprised if this made a significant difference? but I'll take a look.

If you want to open an issue with verbose logs (`-vv`) I'd be happy to look into it. You can set `UV_LOG_CONTEXT=1` to emit timing information.

If you're running Python scripts, I'd recommend using `uv lock --script <path>` to generate a lockfile — we don't do that by default for scripts yet but that will avoid unexpected upgrades.

(I work on uv)

That project looks owned by https://github.com/karpetrosyan who has contributed to httpx but is not the owner / maintainer in question

We want to support that workflow / experience but we want to design it differently because there are a lot of nuances to pip's interface and the pip maintainers have encouraged us to rethink it.

It's also non-trivial to implement because of how our cache design differs from pip's.

Yes! It's in progress — it's already there, just hidden as we iterate on it.

(but as a note, the initial `uv lock` will already attempt to choose the highest possible versions and `uv lock --upgrade` will do so again, requests for `uv upgrade` are generally centered around changing the constraints in the `pyproject.toml`)

It makes `uv run <project-name>` work by default, I expect most projects to need an entry point.

So far the maintenance burden has been quite bearable and extending pip provides us with an opportunity to explore and encourage improvements upstream which are still beneficial to a lot of people.

I hadn't considered that, it's an interesting idea. We generally hold a very high bar for nagging users or gating behaviors. It's possible we'll reach a point someday where this is appropriate! I think you're right that we need to teach people better workflows, but that's happening organically as use of the tool grows and people's first exposure to a Python project is `uv init` and `uv add` rather than `pip`.

Unfortunately dynamic metadata is not efficient for package resolution, so we don't want to encourage it. Instead, we want to design a better workflow for using source control for versioning, but haven't had the time to do so.

Pixi is very focused on the Conda ecosystem and implements a sort of bridge across the Conda and PyPI ecosystems. I am very interested in solving the problems that motivated creation of the Conda ecosystem, but I want to do so by driving forward the PyPI ecosystem so those use-cases can be addressed in _many_ tools instead of just one. This is the main reason you don't see us pursuing Conda package support in uv directly (in the style of Pixi).

There have been several threads about us collaborating or converging, but there are no concrete plans to do so at this time. We share some Rust crates, e.g., Pixi uses our internals to solve the PyPI-side of their dependencies, and we occasionally fix or change things for each other.

Yes! We've been working on this for quite some time as it's one of the worst experiences in Python packaging today.

There are two PEPs that we're involved with that will provide the components for the ecosystem to properly declare metadata in this domain [1] [2].

We're also working on some features in uv, such as the torch backend option [3], to improve the situation in the interim.

We'll be announcing more in this area in the near future!

[1] https://peps.python.org/pep-0817/ [2] https://peps.python.org/pep-0825/ [3] https://docs.astral.sh/uv/guides/integration/pytorch/#automa...

I think we're doing what we should making isolated environments easy to use. I don't think we should take options away from people that need them.

We have the interface because we care a lot about meeting people where they are.

We keep the interface because we care a lot about long-term support and compatibility. I cannot imagine removing it.

We generally want to add features to the top-level interface such that people don't need or want to use the pip-interface because the top-level interface does everything you need, but better.

I don't think it's true at all that "dropping features" makes uv fast. As an author of uv, I think that particular section of the article is way off base.

(I work on uv)

As a note, you can set the default bounds for `uv add` in persistent configuration — no need to provide it every time. See https://docs.astral.sh/uv/reference/settings/#add-bounds

We prefer not to add upper bounds by default because it causes a lot of unnecessary conflicts in the ecosystem. I previously collected some resources on this back when I used Poetry :) see https://github.com/zanieb/poetry-relax#references

I'm a bit confused, `uv run --with $package main --help` should do what you say with very little overhead. We won't reinstall it every time, `--with` environments are stored in the cache and retained. Even if the environment is cached, the dependency is cached and installing from the cache is very fast (<200ms for sure).

Please feel free to open a reproduction with details and we can look into it.

(I work on uv)

fwiw `uv upgrade` is on the roadmap — we just haven't done it yet because it's hard to build a great experience for it (there are far more nuances than people expect) and we're a small team with a lot of priorities.

We do address this in the article! It's defense in depth, not theater.

We audit all of our actions, check if they pull in mutable dependencies, contribute upstream fixes, and migrate off using any action when we can.

(I work at Astral)

A brief note, your numbers are way off here — Astral subsequently raised a Series A and B (as mentioned in the blog post) but did not announce them. We were doing great financially.

(I work at Astral)

Ruff wasn't named after the bird, we just think it's funny that Charlie didn't know it was a bird. He made up the word :)

This is also documented at https://github.com/astral-sh/ty?tab=readme-ov-file#contribut... and https://github.com/astral-sh/ty/blob/main/CONTRIBUTING.md#re...

As noted in the linked issue

> At time of writing, many of the remaining rules require type inference and/or multi-file analysis, and aren't ready to be implemented in Ruff.

ty is actually a big step in this direction as it provides multi-file analysis and type inference.

(I work at Astral)

You're upset that uv doesn't yet support something that no other tool in the ecosystem supports?

I'd love for uv to lock build dependencies, but due to the dynamic nature of Python package metadata it's quite a hard problem. It'll be supported eventually though.

(I work on uv)

It's intentionally distinct from the `uv tool` interface — it won't change `ruff` or `uv tool run` behaviors.

Thanks that's helpful.

Did you try reducing the concurrency limit?