News for hackers. Stuff that matters.
Re:How I think I want to drop modern Python packages into a ... by PaulHoule on Thursday September 17, 2020 @03:41PM (#24509042)
With Python the future is here but it is not evenly distributed. In particular, it's been depressing to see that "any answer at all" has so often won out against "a correct answer".

Java has been a safe place to deploy code to because of its xenophobia. You don't expect Java to be linked into endless C libraries, and Java often doesn't come pre-installed thanks to license issues. (Maybe Larry Ellison is rich because you read "INSTALL JAVA TO EXPERIENCE THE POWER OF JAVA". The consequence is that you got a clean copy of Java and that your clean copy of Java won't break all the crapplets that came with your OS that expected an ancient Java that came with the OS.

Python got a boon from being shipped w/ most Linux distros, but it was also a setback... On Linux "python" became a synonym for "python 2" because otherwise you'd break the system scripts. (Today it drives me nuts that Microsoft added "python" to the past on Windows because Windows was the one operating system where you didn't have to hide the system Python to get a good Python)

"venv" is genius but conda got so much mindshare that many people never realized "venv" could do everything "conda" does except for corrupting internal data structures that force a periodic reinstall.

The consequence is that "best practices" haven't been institutionalized in most places, you have to continuously fight bad practices, and people like the poster are always inventing non-solutions to the problem venv solves.

(e.g. for instance, what about the person who uses a Docker container to keep python environments apart as opposed to one who uses venv?)

Re:How I think I want to drop modern Python packages into a ... by tempay on Thursday September 17, 2020 @04:09PM (#24509342)
For conda it’s an unfair characterisation. Conda is language agnostic and can be (and is) used to ship anything. Packages like GCC/Clang/R will never be accessible for venv. Personally I find it magical to be able to ship complex packages on conda-forge that “just work” on Linux/macOS/Windows/ARM/POWER without requiring users to have special privileges.
Re:How I think I want to drop modern Python packages into a ... by PaulHoule on Thursday September 17, 2020 @04:57PM (#24509860)
I'm tempted to say "so what?"

Here's a specific example. In a previous job I had to work with Tensorflow models of various vintages that depended on different versions of CUDA.

Conda (out of the box) did nothing to help with that problem. I think an early version of Conda might have installed one early version of CUDA, but the elephant in the room as that I would need to install as many versions of CUDA as necessary.

NVIDIA uses a lot of intimidation to fool people as to the nature of the drivers. Specifically, CUDA/CuDNN and such don't depend on kernel drivers or Windows installers, but just require that some huge directories full of userspace libraries be on your path. Thus you can have CUDA version A linked into one conda environment, and CUDA version B linked into another conda environment and it all worked.

That is, it worked when I made my own conda packages (conda wasn't allowed to distribute such package by NVIDIA) and it was astonishingly slow because conda stores packages in BZIP2 files that take forever to unpack and because conda wastes more time with it's symlink machine.

In the end it was possible to solve the problem w/ conda but if I hadn't been distracted by conda I probably would have solved it just as quickly by unpacking the userspace drivers once and just setting the path appropriately.

Re:How I think I want to drop modern Python packages into a ... by marmaduke on Thursday September 17, 2020 @05:13PM (#24510034)
> "conda" does except for corrupting internal data structures

Hold up, what? I have used both for years, how did you corrupt anything?

Re:How I think I want to drop modern Python packages into a ... by PaulHoule on Thursday September 17, 2020 @08:38PM (#24511703)
I had to reinstall every month or so, other devs didn't seem able to maintain a good configuration for more than two weeks.
Re:How I think I want to drop modern Python packages into a ... by marmaduke on Friday September 18, 2020 @02:59PM (#24520616)
You’re describing the consequence, not the cause, after making a pretty strong claim about a package manager a lot of people use daily. Want to pony up that cause?