News for hackers. Stuff that matters.
Re:Potential of the Julia programming language for high ener... by aborsy on Monday December 04, 2023 @08:06AM (#38516757)
Julia is great for numerical simulations. Companies such as Google and Facebook should have supported Julia for machine learning, in my opinion.

Numpy is OK, but verbose and a bit of a pain.

Re:Potential of the Julia programming language for high ener... by ykonstant on Monday December 04, 2023 @08:38AM (#38517030)
> Companies such as Google and Facebook should have supported Julia for machine learning, in my opinion.

Given that ML is becoming so important and Julia has ergonomic handling of numerical code, why don't they explicitly support it? Is it inertia?

Re:Potential of the Julia programming language for high ener... by roflmaostc on Monday December 04, 2023 @09:08AM (#38517369)
From my point, there is still too much change around automatic differentiation libraries.

There is Zygote.jl, which is used in Flux.jl. However, it's more in maintenance mode. At some point, Diffractor.jl was hyped but it didn't take off yet. And then there is Enzyme.jl which people hype now.

But for me as a user, it's not clear what I should really do to make my code well differentiable for those libraries.

If you stick with torch, jax or tensorflow, everything seems to work better regarding AD.

[0]: https://github.com/FluxML/Flux.jl [1]: https://github.com/FluxML/Zygote.jl [2]: https://github.com/JuliaDiff/Diffractor.jl [3]: https://github.com/EnzymeAD/Enzyme.jl

Re:Potential of the Julia programming language for high ener... by chriskanan on Monday December 04, 2023 @09:03AM (#38517311)
The developers of Pytorch put out an article about this a few years ago and they admitted Julia would make sense from a pure ML perspective but the weight of the Python ecosystem was too critical.

I wonder if GPT-4 could be used to efficiently start porting more stuff over to Julia. I feel like Python is really a poor fit for deep learning. Julia is much more pleasant.

Re:Potential of the Julia programming language for high ener... by disgruntledphd2 on Monday December 04, 2023 @11:50AM (#38519619)
The only reason PyTorch exists is that Yann LeCunn couldn't convince people at Facebook to use the original Lua implementation.

So yeah, I'd be surprised if FB ever moved away from Python.

Re:Potential of the Julia programming language for high ener... by alechammond on Monday December 04, 2023 @10:35AM (#38518537)
Note that Meta recently added a Julia ruleset to its buck2 prelude set. An important step toward working with Julia on prod environments.

https://github.com/facebook/buck2/tree/main/prelude/julia

Re:Potential of the Julia programming language for high ener... by bmitc on Monday December 04, 2023 @08:41AM (#38517063)
They have a 100,000 Python, C++, and Java devs. My assumption is they don't care, i.e., inertia like you said.

The big mistake was Microsoft not having .NET cross-platform from the beginning and not pitching F# as a Python alternative.

Re:Potential of the Julia programming language for high ener... by soulbadguy on Monday December 04, 2023 @12:14PM (#38519979)
> F# as a Python alternative

To play devil advocate, i think when F# was out (and still being developed), the Zeitgeist was really enamored with the idea of dynamic language, and the envisioned productivity gain from them. So F# was really too early to the dance.

Since then, things have change quite a bit with most major dynamic language adding a proto-type system.

Right now, i think we understand better what made language such as python and ruby so popular and are able to design better statically typed version with better tooling around them ( rust, zig, julia etc...)

Re:Potential of the Julia programming language for high ener... by _Wintermute on Monday December 04, 2023 @10:35AM (#38518540)
Instead we got Tensorflow for ... Swift, which nobody asked for.
Re:Potential of the Julia programming language for high ener... by raccoonDivider on Monday December 04, 2023 @01:56PM (#38521438)
Apple probably asked and paid for it. Isn't there potential in accelerating LLMs and transformers on smartphones? Most of what they are good at fits the smartphone use case well.
Re:Potential of the Julia programming language for high ener... by pjmlp on Tuesday December 05, 2023 @08:03AM (#38530185)
Not at all, it was more related to well known Swift folks at Google.
Re:Potential of the Julia programming language for high ener... by Joel_Mckay on Monday December 04, 2023 @08:41AM (#38517058)
You didn't look hard enough.

ML on CUDA is very much part of Julia already, and Meta seems into that lib as well. =)

Re:Potential of the Julia programming language for high ener... by jjoonathan on Monday December 04, 2023 @12:13PM (#38519969)
Is it good? PyCUDA was so bad (as of 2018) that after 3 major bugfixes from me and no end in sight it still almost sank a major project until I gave up on it and went with native CUDA.
Re:Potential of the Julia programming language for high ener... by empyrrhicist on Monday December 04, 2023 @01:39PM (#38521180)
Anecdotally, the Julia approach certainly seems... nicer via e.g. KernelAbstractions etc. I don't know if the performance/flexibility is quite there vs. native CUDA, but going from zero to GPU kernel programming in Julia is as close to painless as I've ever seen, especially given how modular everything is (for example, you can use OffsetArrays.jl directly in GPU kernel code).
Re:Potential of the Julia programming language for high ener... by Joel_Mckay on Monday December 04, 2023 @06:00PM (#38524409)
"Is it good?"

Compared to the CUDA dumpster fire at a cat food factory, it is often trivial and nearly transparent syntax for the users familiar with ML.

Really depends on the use-case =)

The conventional options are fairly well documented:

https://sciml.ai/

https://fluxml.ai/Flux.jl/stable/gpu/

https://github.com/SciML/DiffEqFlux.jl

https://github.com/alan-turing-institute/MLJ.jl

Re:Potential of the Julia programming language for high ener... by raccoonDivider on Monday December 04, 2023 @01:59PM (#38521485)
I tried using the latest stable Flux.jl with the latest stable Julia interpreter about a year ago, and most of it didn't work. Some of the simplest model usage examples led to a traceback with dozens of frames, ending in an incomprehensible internal error that I couldn't possibly understand as a beginner.

It seems to me that Julia is still moving too fast for its own good. At some point, something needs to stabilize for it to be worth adopting.

Re:Potential of the Julia programming language for high ener... by Joel_Mckay on Monday December 04, 2023 @08:39AM (#38517040)
Numpy is not really "OK" for everyone, it is still classed as a functional compatibility layer for a fundamentally broken language paradigm.

People often conflate popularity with good design. Have a great Monday =)

Re:Potential of the Julia programming language for high ener... by bmitc on Monday December 04, 2023 @08:42AM (#38517073)
Are there any good references for what a library like Numpy should look like?
Re:Potential of the Julia programming language for high ener... by qart on Monday December 04, 2023 @10:33AM (#38518503)
Yes, the paper in the original post has a few examples. Julia uses expressions that are based on operators we use in regular arithmetic, even in plain Python. So, just ordinary operators instead of np.matmul, np.ling.expm, etc.
Re:Potential of the Julia programming language for high ener... by jampekka on Tuesday December 05, 2023 @11:33AM (#38533097)
Matrix multiplication works with the @ operator, e.g. A@B. (Or with the * operator if defined as np.matrix, but nobody uses that and probably shouldn't). For matrix exponentials you need scipy.linalg.expm, but there is nothing in Python preventing why it couldn't be done with e.g. e**M if wanted (or even e^M if wanted but probably shouldn't). You can even implement it yourself in a few lines.

You don't seem to know much about what you're criticizing.

Re:Potential of the Julia programming language for high ener... by qart on Tuesday December 05, 2023 @01:56PM (#38535338)
Lol Good luck working in a project where people have decided to create wrappers like you suggested.
Re:Potential of the Julia programming language for high ener... by jampekka on Tuesday December 05, 2023 @02:07PM (#38535464)
For expm? The matmul as @ is numpy standard and used widely. If you have something that needs expm so much it needs to be an operator, good luck working in a project where people can't understand one-line class definition.
Re:Potential of the Julia programming language for high ener... by qart on Tuesday December 05, 2023 @02:32PM (#38535800)
I simply refuse to work with code written by people who have had a hundred such harebrained ideas. No luck needed, but thanks!
Re:Potential of the Julia programming language for high ener... by jampekka on Tuesday December 05, 2023 @03:53PM (#38536757)
Why is it any more or less harebrained for the language devs to overload an operator in the core?
Re:Potential of the Julia programming language for high ener... by eigenspace on Monday December 04, 2023 @09:14AM (#38517461)
Julia!
Re:Potential of the Julia programming language for high ener... by Joel_Mckay on Monday December 04, 2023 @09:07AM (#38517353)
Most of Erlang, Julia, and Go... As they tended to re-factor dependencies and trend toward a homogeneous distributed ecosystem.

Python is the core issue rather than specifically Numpy, as it was bodged on (SWIG) to try to fix real use-cases... much like how 30 years of bodged on GPU mailbox structures made people biased to implement ridiculous solutions that necessitated ridiculous software paradigms.

I don't think any one person has the will and resources to resolve the core problem. However, Julia will likely also initially bind the same nonsense for awhile due to compatibility needs, but users tend to re-factor nonsense out of the ecosystem with better native solutions over time.

"All software is terrible, but some of it is useful" =)

Re:Potential of the Julia programming language for high ener... by bmitc on Monday December 04, 2023 @07:48PM (#38525577)
I am specifically looking for Numpy-like libraries and designs, not languages that have generally well-designed libraries. For example, Elixir has Nx.

It seems to me that, even given Python's constraints as a language, that a nicer wrapper could have been developed in Numpy that still called out to bindings to do the heavy work.

> functional compatibility layer for a fundamentally broken language paradigm

What did you mean by this? Is it mainly the fact that Python is a bit an undesigned language in general that calls into unmanaged languages like C/C++ and Fortran in its libraries?

Re:Potential of the Julia programming language for high ener... by Joel_Mckay on Monday December 04, 2023 @08:41PM (#38525945)
"I am specifically looking for Numpy-like libraries and designs"

Why would one bottleneck a design with polyglot stacks even before a single line of code was implemented? This sounds like naive nonsense.

"What did you mean by this? ... Python"

Python was never designed to handle threads or parallelism properly, and has performance issues Numpy tries to address though its wrapped C/C++ libraries.

Python became 30 years of spiral development, and implodes into a new implementation every so often. Depending on the use-case it may prove appropriate, but never optimal. =)

Re:Potential of the Julia programming language for high ener... by bmitc on Monday December 04, 2023 @09:20PM (#38526215)
I was wondering about libraries that do Numpy-like things, not libraries implemented like Numpy.
Re:Potential of the Julia programming language for high ener... by Joel_Mckay on Monday December 04, 2023 @11:09PM (#38527000)
There are many native packages and wrapper scaffold libraries like Numpy. It will depend on your problem domain, but Julia often transparently supports its broadcast operator on most core data-structure math ("using LinearAlgebra" and the like).

https://gist.github.com/AlexanderFabisch/6343090 (small feature comparison list, but ignore the incorrect opinions)

https://sciml.ai/

https://fluxml.ai/Flux.jl/stable/gpu/

https://alan-turing-institute.github.io/MLJ.jl/dev/about_mlj...

Re:Potential of the Julia programming language for high ener... by dist-epoch on Monday December 04, 2023 @09:06AM (#38517344)
Use of Python in ML was bottom-up, not dictated by the CEO.
Re:Potential of the Julia programming language for high ener... by agumonkey on Monday December 04, 2023 @02:38PM (#38522047)
Won't there be too much cultural inertia for all the people who invested their brain into numpy ?