News for hackers. Stuff that matters.
Re:How to create a Python package in 2022 by lmeyerov on Thursday July 28, 2022 @01:52AM (#32260879)
We've been fine-ish with classic setup.py/setup.cfg + gha for publishing to pypi. But as we do OSS data science (gpu graph ai + viz), where conda is typical nowadays...

... Have to admit: We recently ended up contracting conda packaging out because it was nowhere near clear enough to make sense for our core team to untangle. Would love to see a similar tutorial on a github flow packaging & publishing to conda. Still no convinced we're doing it right for subtleties like optional dependencies: equivalent of `pip install graphistry` vs `pip install graphistry[ai]` vs `graphistry[umap-learn]`, etc.

Re:How to create a Python package in 2022 by plonk on Thursday July 28, 2022 @02:37AM (#32261130)
We do a lot of deep learning and image processing and pip works much better for us. PyTorch makes wheels that contain all reauired DLLs on all systems. Maybe conda isn’t needed anymore.
Re:How to create a Python package in 2022 by notemaker on Thursday July 28, 2022 @02:45AM (#32261181)
I think of conda as jupyter notebooks. It's ok to experiment with and get something running quickly, but it's a bad idea to use for production code.

Personally I never use either of these tools.

Re:How to create a Python package in 2022 by quantumduck on Thursday July 28, 2022 @02:42AM (#32261157)
Conda let's you install a specific cuda version directly in a virtual environment with one click though. It's really useful when you have to switch between multiple PyTorch versions and convinient in general imo.
Re:How to create a Python package in 2022 by plonk on Thursday July 28, 2022 @01:36PM (#32267685)
The PyTorch wheel comes with its own CUDA DLLs, they can’t be shared with other libraries but they won’t be your problem to install that way.
Re:How to create a Python package in 2022 by werewolf on Thursday July 28, 2022 @03:20AM (#32261394)
I found conda-forge community quite helpful here [1]. They make feedstock repositories based on templates that cover a lot of automation. Their bots pickup updated packages in pypi and automatically file merge requests, run tests and even merge updates if tests pass successfully. Basically you only need to maintain your recipe here and there when your dependencies change.

1. https://conda-forge.org/docs/user/introduction.html

Re:How to create a Python package in 2022 by lmeyerov on Thursday July 28, 2022 @09:26AM (#32263968)
Yep, that was one of the better docs!

The contractor still took a couple weeks to figure out and get up. I assumed it'd be one evening for initial bulk as we already had setup.cfg etc, but after searching conda tutorials.. not surprised. Our ~final meta file is pretty simple, so no idea why the docs are so indirect.

Re:How to create a Python package in 2022 by RojerGS on Thursday July 28, 2022 @03:12AM (#32261336)
Oh, I see... I personally barely use conga and I have no idea how that is done. I don't think I'll write any blog article like that any time soon :( Maybe you could do it!