News for hackers. Stuff that matters.
Re:Ask HN: Machine Learning Engineers, what do you do at work? by posix_monad on Saturday June 08, 2024 @07:37AM (#40616916)
Python's dominance is holding us back. We need a stack with a more principled approach to environments and native dependencies.
Re:Ask HN: Machine Learning Engineers, what do you do at work? by llm_trw on Saturday June 08, 2024 @08:33AM (#40617124)
Here's what getting PyTorch built reproducibly looks like: https://hpc.guix.info/blog/2021/09/whats-in-a-package/

Since then the whole python ecosystem has gotten worse.

We are building towers on quicksand.

It's not about python, it's about people who don't care about dependencies.

Re:Ask HN: Machine Learning Engineers, what do you do at work? by hkt on Saturday June 08, 2024 @10:39AM (#40617821)
Dependency management is just.. hard. It is one of the things where everything relies upon it but nobody thinks "hey, this is my responsibility to improve" so it is left to people who have the most motivation, academic posts, or grant funding. This is roughly the same problem that led to heartbleed for OpenSSL.
Re:Ask HN: Machine Learning Engineers, what do you do at work? by manusachi on Saturday June 08, 2024 @01:45PM (#40619148)
Do you know what other ecosystem comes closest to the existing in Python? I've heard good things about Julia.

13 years ago when I was trying to explore the field R seemed to be the most popular, but looks like not anymore. (I didn't get into the field, and do just a regular SWE, so I'm not aware of the trends).

There is also a lot of development in Elixir ecosystem around the subject [1].

[1](https://dashbit.co/blog/elixir-ml-s1-2024-mlir-arrow-instruc...)

Re:Ask HN: Machine Learning Engineers, what do you do at work? by posix_monad on Monday June 10, 2024 @04:30AM (#40631385)
I don't think the ML community has an appetite for learning a different language.

There are Microsoft-backed F# bindings for Spark and Torch, but no one seems interested. And this is despite a pretty compelling proposition of lightweight syntax, strong typing, script-ability and great performance.

The answer will probably be JavaScript.

Everyone already knows the language - all that's missing is operator overloading and a few key bindings.

Re:Ask HN: Machine Learning Engineers, what do you do at work? by soulbadguy on Monday June 10, 2024 @09:58AM (#40633753)
> There are Microsoft-backed F# bindings for Spark and Torch > this is despite a pretty compelling proposition of lightweight syntax, strong typing, script-ability and great performance.

For exactly the reasons your mentioned, i feel like F# would have been the perfect match for both MLE/ETL(spark pipeline) work and some of the deep learning/graph modelisation such as pytorch. Saddly, even from MSFT, the investment in F# as dried up

Re:Ask HN: Machine Learning Engineers, what do you do at work? by posix_monad on Tuesday June 11, 2024 @11:51AM (#40647757)
Not sure what you mean here - Microsoft are actively working on F# and the .NET Spark and Torch bindings.