Ask HN: Machine Learning Engineers, what do you do at work?

Re:Ask HN: Machine Learning Engineers, what do you do at work? by SushiHippie on Friday June 07, 2024 @02:56PM (#40611739)

Can recommend using conda, more specifically mambaforge/micromamba (no licensing issues when used at work).

This works way better than pip, as it does more checks/dependency checking, so it does not break as easily as pip, though this makes it definitely way slower when installing something. It also supports updating your environment to the newest versions of all packages.

It's no silver bullet and mixing it with pip leads to even more breakages, but there is pixi [0] which aims to support interop between pypi and conda packages

[0] https://prefix.dev/

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by tasuki on Friday June 07, 2024 @03:22PM (#40612001)

I had a bad experience with Conda:

- If they're so good at dependency management, why is Conda installed through a magical shell script?

- It's slow as molasses.

- Choosing between Anaconda/Miniconda...

When forced to use Python, I prefer Poetry, or just pip with freezing the dependencies.

The Python people probably can't even imagine how great dependency management is in all the other languages...

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by tedivm on Friday June 07, 2024 @03:47PM (#40612225)

I absolutely hate conda. I had to support a bunch of researchers who all used it and it was a nightmare.

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by akkad33 on Friday June 07, 2024 @03:53PM (#40612290)

Mamba/micromamba solves the slowness problem of conda

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by epoxia on Friday June 07, 2024 @04:09PM (#40612459)

To add. Conda has parallelized downloads and is faster. Not as fast as mamba, but faster than previously. pr merged sep 2022 -> https://github.com/conda/conda/pull/11841

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by SushiHippie on Friday June 07, 2024 @09:24PM (#40614559)

Yeah, I agree, maybe I should have also mentioned the bad things about it, but after trying many different tools that's the one that I stuck with, as creating/destroying environments is a breeze once you got it working and the only time my environment broke was when I used pip in that environment.

> The Python people probably can't even imagine how great dependency management is in all the other languages...

Yep, I wish I could use another language at work.

> Choosing between Anaconda/Miniconda...

I went straight with mamba/micromamba as anaconda isn't open source.

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by semi-extrinsic on Saturday June 08, 2024 @02:21AM (#40615682)

rye and uv, while "experimental", are orders of magnitude better than poetry and pip IMHO.

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by davedx on Saturday June 08, 2024 @02:20AM (#40615674)

Yes I started with conda I think and ended up switching to venv and I can’t even remember why, it’s a painful blur now. It was almost certainly user error too somewhere along the way (probably one of the earlier steps), but recovering from it had me seriously considering buying a Linux laptop.

This happened about a week ago

[ Parent | on HN ]

Re:Ask HN: Machine Learning Engineers, what do you do at work? by reportgunner on Sunday June 09, 2024 @11:21AM (#40625031)

Whenever anyone recommends conda I automatically assume they don't know much about python.

[ Parent | on HN ]