News for hackers. Stuff that matters.
Re:How to make a Python package in 2021 by staticassertion on Wednesday April 07, 2021 @11:20PM (#26734747)
TBH as someone trying to use Python professionally it is extremely frustrating that basic things with regards to package management are something you have to iterate towards, as opposed to just being obvious and default.
Re:How to make a Python package in 2021 by TylerE on Wednesday April 07, 2021 @11:30PM (#26734817)
One thing that has become clear to me, from playing around a bit with go, rust, and nim, is that it is astonishingly better when the language has exactly one set of tools, that everyone uses.

Even if that set of tools is kinda crappy (Glances at go), it's just so nice to not have all the bikeshedding and just get on with it.

Re:How to make a Python package in 2021 by CraigJPerry on Thursday April 08, 2021 @05:07AM (#26736641)
I’m not familiar enough with Go but at first glance the go mod stuff seemed pretty decent.

E.g. more flexible than Cargo in that you could have a large codebase with two different versions of a dependency isolated to their compile units allowing you to gradually adopt a breaking change dependency in a large code base. I was kinda bowled over with that feature (the isolation part is key).

For python I’m finding poetry much more ergonomic than pipenv, it’s not just the speed difference it’s the convenience of one tool which aligns with what you’re saying although the existence of poetry doesn’t delete historical aberrations in Python’s long history.

Re:How to make a Python package in 2021 by zomglings on Wednesday April 07, 2021 @11:25PM (#26734788)
I sympathize. It is unfortunate that the python community never settled around a tool like leiningen for clojure or cargo for rust or npm for node.

What we saw with npm was the entire community iterating towards a feature set and everyone reaping the benefits automatically with npm updates. package-lock.json is a good example of this.

Re:How to make a Python package in 2021 by brundolf on Wednesday April 07, 2021 @11:51PM (#26734923)
Worth noting is that cargo and npm weren't "settled around"; they were developed and presented, from the beginning, alongside the relevant compiler and runtime. There was never a question; the batteries were included.

Leiningen is the weird one where people did actually settle fairly well around an unofficial solution in the absence of an official one. I think the norm with languages that forego official tooling is closer to what we've seen in Python.

Re:How to make a Python package in 2021 by xapata on Thursday April 08, 2021 @12:29AM (#26735098)
The Python community has considered an "official" packaging tool in the past, but in those conversations found that the community had too many preferences to find a good compromise. That's the trouble with having a highly diverse set of uses and integrations, and lots of legacy.

If you're curious, the email threads about Conda and defining wheels are interesting.

Re:How to make a Python package in 2021 by staticassertion on Thursday April 08, 2021 @12:42AM (#26735157)
I feel like the entire point of a BDFL is that they can just ignore this sort of thing and make the hard call, but it never happened.
Re:How to make a Python package in 2021 by divbzero on Thursday April 08, 2021 @01:05AM (#26735267)
Maybe it could still happen? It seems like a super high value challenge that the BDFL could take on: build out the official set of tools (setup.py, twine, virtualenv, pip) to support features that make people seek out alternatives (pyproject.toml, poetry, flit, conda, pyenv, pipenv).
Re:How to make a Python package in 2021 by jolux on Thursday April 08, 2021 @01:49AM (#26735486)
I realize this is controversial but from reading the docs I really thought Pipenv was the official solution. Took me a while to realize this wasn't the case.
Re:How to make a Python package in 2021 by divbzero on Thursday April 08, 2021 @02:01AM (#26735553)
I went through the same progression, thinking pipenv was the official solution before deciding it wasn’t. Then, just now, I realized that pipenv [1] is currently owned by the Python Packaging Authority (PyPA) who also owns pip [2] and virtualenv [3]. I don’t know the right answer but this illustrates the confusion of not coalescing around an official solution.

[1]: https://github.com/pypa/pipenv

[2]: https://github.com/pypa/pip

[3]: https://github.com/pypa/virtualenv

Re:How to make a Python package in 2021 by toyg on Thursday April 08, 2021 @05:11AM (#26736670)
What happened was that Kenneth Reitz socially-engineered his way into the PyPA to get his tool blessed. The community lashed out (since the tool had obvious shortcomings and a somewhat dubious development process) and recommendations were softened. Eventually the PyPA had to take over pipenv when Reitz had other issues, and they are now forever burdened with what is a bit of a dud.
Re:How to make a Python package in 2021 by xapata on Thursday April 08, 2021 @02:08AM (#26735600)
As I understand it, that was to prevent bad behavior rather than to signal approval.
Re:How to make a Python package in 2021 by staticassertion on Thursday April 08, 2021 @01:46AM (#26735464)
I think the time has passed, both in terms of there no longer being a BDFL, as well as Python passing a point where the call can be made.
Re:How to make a Python package in 2021 by divbzero on Thursday April 08, 2021 @01:52AM (#26735503)
Sadly I bet you’re right on this. My small sliver of hope is that Guido van Rossum’s ideas to “make using Python better” [1] include better packaging.

[1]: https://news.ycombinator.com/item?id=25071847

Re:How to make a Python package in 2021 by takluyver on Thursday April 08, 2021 @04:31AM (#26736412)
There are quite a lot of people working on better packaging (look at the Python discourse forum, for instance), but this is not really a topic that Guido has got involved in, at least in the time I've been paying attention.

But with or without a BDFL, one packaging tool to rule them all is a pretty tall order. The needs of a package like scipy, which incorporates C & Fortran code, are pretty different from something like requests. And different communities using Python have their own entrenched tools and techniques. It takes more than someone saying "Foo is officially blessed" to shift that, even if everyone respects the person saying that.

Re:How to make a Python package in 2021 by arthurcolle on Thursday April 08, 2021 @02:00AM (#26735550)
Is there a new BDFL? I remember reading this half a year ago or so https://hub.packtpub.com/why-guido-van-rossum-quit/
Re:How to make a Python package in 2021 by xapata on Thursday April 08, 2021 @12:55AM (#26735225)
On the whole, Guido's career as a BDFL was astoundingly effective. Maybe he made the right call. It'd have been a terrible idea to alienate the science community just when data science was taking off as a field.
Re:How to make a Python package in 2021 by staticassertion on Thursday April 08, 2021 @01:51AM (#26735500)
I'm not disagreeing, but I am curious if you have anything specific you would point to with regards to Guido's role being successful. I'm just ignorant really, it's not intended to be a leading question at all.
Re:How to make a Python package in 2021 by xapata on Thursday April 08, 2021 @02:01AM (#26735557)
Too many things to answer here. The easiest is to point to the popularity of the language. He's made decisions I disagree with, but I can't argue with success.
Re:How to make a Python package in 2021 by neolog on Thursday April 08, 2021 @02:04AM (#26735573)
I don't know how much the original author had to do with Python's success in the last 20 years. The success of Python in data science is because of NumPy/Scipy/Pandas. Things like packaging that needed leadership never got any.
Re:How to make a Python package in 2021 by xapata on Thursday April 08, 2021 @02:09AM (#26735609)
The leadership was that the science community would benefit from a science-specific tool, like Conda, and that making one ring to rule them all would be too difficult.

Also, yes, Guido and many other early contributers have been been active for 30ish years.

Re:How to make a Python package in 2021 by neolog on Thursday April 08, 2021 @08:55PM (#26745637)
Conda isn't really science specific, just like pip isn't webdev specific.
Re:How to make a Python package in 2021 by xapata on Friday April 09, 2021 @11:46AM (#26752046)
The result isn't, but the motivation was. And have you noticed who is using which tool?
Re:How to make a Python package in 2021 by toyg on Thursday April 08, 2021 @04:57AM (#26736588)
> The success of Python in data science is because of NumPy/Scipy/Pandas.

And NumPy/Scipy/Pandas exist (and are successful) because...? /s

Re:How to make a Python package in 2021 by regularfry on Thursday April 08, 2021 @10:32AM (#26739185)
Rubygems, and then Bundler, followed the same pattern. Neither was batteries-included, both were unofficial community efforts. Bundler directly influenced cargo.
Re:How to make a Python package in 2021 by CraigJPerry on Thursday April 08, 2021 @05:10AM (#26736663)
Yarn / npm are still in competition today i think?

The js tooling is particularly immature, e.g. fake packages along with typo squatting is rife. Compare with boring old maven where that’s not been an issue in >10 years at this point.

Re:How to make a Python package in 2021 by staticassertion on Wednesday April 07, 2021 @11:28PM (#26734803)
I completely agree. Elsewhere we see sanity by default, in Python there's still open questions and hopeful but early stage projects.
Re:How to make a Python package in 2021 by marcinzm on Wednesday April 07, 2021 @11:32PM (#26734829)
The issue is that every time the community settles on a tool a new tool is made to fix the issues with the old tool rather than just refactoring the old tool.
Re:How to make a Python package in 2021 by mrweasel on Thursday April 08, 2021 @01:54AM (#26735512)
I think the Python community have largely settled on pip, just like the JavaScript community have mostly opted for npm.

Personally I much prefer pip over npm, it feels much more polished.

Re:How to make a Python package in 2021 by type0 on Friday April 09, 2021 @12:34PM (#26752656)
but pip isn't enough and not exactly user friendly
Re:How to make a Python package in 2021 by cixinlui on Thursday April 08, 2021 @05:10AM (#26736665)
I know Python much better than I do rust or node, but I think the Python design decision was decent here: you can use different tools, but all of them should put the configuration in pyproject.toml. That file has fields which are universal and others which can depend on the exact tooling used. So build tools, repos and so on can get the info they need and at least potentially, do the right thing for code packages with different tools.
Re:How to make a Python package in 2021 by blondin on Wednesday April 07, 2021 @11:53PM (#26734934)
yea, it is. but it's a sane thing to do. recommending poetry for a beginner is a bad idea. (nothing against that package).

python is a mature, old, software system. three times older than go or rust. way older than zig. these modern languages have learned from the field as a whole and implemented tools that people are taking for granted nowadays.

as with any mature software system, people & companies have established their preferred ways of doing things. it's going to be hard to have the language council dictate ways of doing things.

i would recommend going simple and using the standard set of modules until poetry or whatever gets into it.

Re:How to make a Python package in 2021 by feu on Thursday April 08, 2021 @05:26AM (#26736765)
Recommending the standard set of modules is the opposite of "going simple." Poetry removes a lot of the complexity and user-unfriendliness inherent in the previous set of standard modules. For any Python beginner coming from another popular language Poetry is likely going to be very similar to the dependency and package management in the language they're coming from. Python's standard set of modules sticks out like a sore thumb when compared to the tools in other popular languages.
Re:How to make a Python package in 2021 by staticassertion on Thursday April 08, 2021 @12:43AM (#26735165)
Well, Python did set up a standard - pyproject.toml. And it's what poetry uses. It's just, uh, "light".
Re:How to make a Python package in 2021 by goerz on Thursday April 08, 2021 @01:11AM (#26735287)
It's also not ready yet, missing critical features like editable installs. Right now, you still need a shim setup.py. Until pyproject.toml can actually replace setupy.py, I see little incentive to start using it: It's just one more file to add. The one exception is if the package actually has build requirements, e.g. for Cython modules.
Re:How to make a Python package in 2021 by zo1 on Thursday April 08, 2021 @02:36AM (#26735774)
The only reason we use pyproject.toml is because of the stupid black formatter that refuses to support setup.cfg, which every other python tool under the sun supports.
Re:How to make a Python package in 2021 by goerz on Thursday April 08, 2021 @02:52PM (#26742347)
Yeah, it's annoying. But Black has so little configuration that I'm ok with hardcoding command-line parameters in my Makefile/tox.ini/precommit hooks (`--skip-string-normalization --line-length 79`)
Re:How to make a Python package in 2021 by mixmastamyk on Thursday April 08, 2021 @04:22AM (#26736349)
There are other formatters, some may even be better.
Re:How to make a Python package in 2021 by cixinlui on Thursday April 08, 2021 @05:12AM (#26736682)
I don't recognize this. I use poetry all the time, without setup.py and the local installs are editable. I have published half a dozen packages which don't have a setup.py and they all work fine.
Re:How to make a Python package in 2021 by neolog on Thursday April 08, 2021 @08:01PM (#26745274)
You can't install a poetry package editably into another environment. It's really a missing feature in the pyproject spec. There's an open issue about it under one of the pypa repos. Someone just needs to do the work of implementing it in pip/setuptools.
Re:How to make a Python package in 2021 by neolog on Thursday April 08, 2021 @05:40PM (#26744106)
> recommending poetry for a beginner is a bad idea.

Strongly disagree. There are so many footguns with low-level tools like pip that I can't recommend it to anybody but an expert (but an expert doesn't need my recommendation anyway).