Guido van Rossum joins Microsoft

Re:Guido van Rossum joins Microsoft by short_sells_poo on Thursday November 12, 2020 @01:08PM (#25072704)

This is probably a hard problem, but the dependency graph resolution in conda is a thing of nightmares. It is so appallingly slow even for relatively menial tasks that updating environments becomes an exercise in frustration.

I'm unsure what are the core issues, but in my experience cargo was always pretty quick to use and if it fails, it fails fast.

Conda on the other hand is slow for the simple cases, and if the graph becomes complex it will just churn for 15 minutes before throwing it's hands in the air and giving up with some cryptic error.

I suspect it comes back to the fact that packaging and dependency management was thought about upfront for Rust and the whole ecosystem was built well from the get go?

[ Parent | on HN ]

Re:Guido van Rossum joins Microsoft by steveklabnik on Thursday November 12, 2020 @01:50PM (#25073319)

I've been meaning to write a blog post on this topic. That is, on the various speeds of various package managers. I don't know a ton about Conda, but when I was looking into Poetry, one of the core issues is: https://python-poetry.org/docs/faq/#why-is-the-dependency-re...

> This is due to the fact that not all libraries on PyPI have properly declared their metadata and, as such, they are not available via the PyPI JSON API. At this point, Poetry has no choice but downloading the packages and inspect them to get the necessary information. This is an expensive operation, both in bandwidth and time, which is why it seems this is a long process.

Cargo doesn't need to do anything with packages directly to do its job; everything it needs is in the index. This makes it pretty fast.

[ Parent | on HN ]

Re:Guido van Rossum joins Microsoft by estebank on Thursday November 12, 2020 @02:07PM (#25073557)

>> This is due to the fact that not all libraries on PyPI have properly declared their metadata and, as such, they are not available via the PyPI JSON API. At this point, Poetry has no choice but downloading the packages and inspect them to get the necessary information. This is an expensive operation, both in bandwidth and time, which is why it seems this is a long process.

This sounds like something that could be done server side, either by PyPI or another entity and expose it through a new API endpoint, instead of doing it on every Python developer's machine.

[ Parent | on HN ]

Re:Guido van Rossum joins Microsoft by steveklabnik on Thursday November 12, 2020 @02:21PM (#25073753)

The core issue is that setup.py can be non-deterministic, so them doing it server side may not give the right results.

https://dustingram.com/articles/2018/03/05/why-pypi-doesnt-k...

[ Parent | on HN ]

Re:Guido van Rossum joins Microsoft by stevesimmons on Thursday November 12, 2020 @01:52PM (#25073358)

conda taking 15+ mins to resolve dependencies is by far its biggest weakness.

[ Parent | on HN ]

Re:Guido van Rossum joins Microsoft by benhurmarcel on Thursday November 12, 2020 @04:55PM (#25075629)

If it doesn't fail to "solve the environment" after 30 minutes that's already a win.

[ Parent | on HN ]