News for hackers. Stuff that matters.
Re:Nx: Multi-dimensional tensors Elixir lib with multi-stage... by elcritch on Thursday February 18, 2021 @05:49AM (#26178330)
Great initiative!

One bit that sticks out to me is that the "standard" math functions like `exp` or `ln` aren't imported despite `defn` being quite maths specific. It seems that they could be imported as default functions inside the scope of `defn`. That leads me to a more important syntax question though.

Do you have any thoughts or plans on operator syntax for elementwise operations vs matrix level operations? One big "win" for numerical Python was having `@` become an infix operator specifically for matrix dots [0, 1]. The syntax in Julia (which follows Matlab) is to have most math operators also have an element wise version. It's generalized to "broadcast" operations in Julia [2]. R appears to have ``, `%%, `%o%` infix operators for elementwise, inner, and outer matrix multiplication.

Really writing any degree of matrix (ahem Tensor) maths and equations becomes very tedious and illegible if you have to use function calls to distinguish elementwise vs matrix operations.

The pipe operator really does help significantly, even with simpler stats oriented code (I've tried a few different formats [4]). Still it's still harder to read if you're familiar with "broadcast" syntax from other mathematics oriented languages.

0: https://alysivji.github.io/python-matrix-multiplication-oper... 1: https://www.python.org/dev/peps/pep-0465/ 2: https://julialang.org/blog/2018/05/extensible-broadcast-fusi... 3: https://www.statmethods.net/advstats/matrix.html 4: https://github.com/elcritch/matrex_numerix/blob/5835a9b477d8...

Re:Nx: Multi-dimensional tensors Elixir lib with multi-stage... by josevalim on Thursday February 18, 2021 @08:57AM (#26179829)
Hi, those are great points and exactly the type of discussions I expect to have with the greater Elixir community over the next months. I will answer them with my current perspective.

Given that Nx has its own numerical kernel, we can import whatever we want. :) I have decided to keep the imported keywords to be a subset of the Elixir's built-in Kernel but there is nothing stopping us from changing that. For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too.

Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :) If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream.

Thank you!

Re:Nx: Multi-dimensional tensors Elixir lib with multi-stage... by elcritch on Thursday February 18, 2021 @03:30PM (#26185056)
Thanks for the response! Glad to hear it's on your radar and ready to foster those discussions. :-)

> For example, if exp and ln are all automatically imported in Julia, Python, etc, then it is most likely more than enough to justify them being imported in `defn` too.

Matlab and Julia import lots of math defaults because that's their primary target. In Python it varies, and it's pretty common to use `np` to avoid cluttering the namespace. Though partly it's because there isn't a good way (AFAICT) to make a scope with default custom namespace like `defn` gives you! It's a bit of taste really though. I'd suggest it's worth considering.

> Regarding custom operators, Elixir has a limited set of operators which have historically been expanded on demand. So it is a matter of looking at other communities, find what is the closest thing to a standard, and send a proposal to the language. :)

Well now, you give me way too many ideas. ;) I'll at least post some issues on the Nx tracker. My initial thought would be that perhaps a `<>` operator for "dot" operation would fit in with Elixir's other operators and is comparable to R's `%%` operators so there's a similarity and precedence. If it were low risk to add then it'd be a pretty decent option.

It wouldn't solve the whole elementwise issue though. but with syntax trees there could be ways to use Einstein notation instead [1]. That'd be fun to play with!

> If you are interested in driving this, you are welcome to submit an initial discussion comparing other languages and an initial set of operators on the Nx issues tracker, then we can discuss it upstream.

I'll create an issue on Nx project and basically repost my thoughts there. Then see what other people are thinking!

1: https://en.wikipedia.org/wiki/Einstein_notation

Re:Nx: Multi-dimensional tensors Elixir lib with multi-stage... by wolf4earth on Friday February 19, 2021 @12:55AM (#26190201)
Just FYI `<>` is already the string concatenation operator. Just in case you're not aware. :)
Re:Nx: Multi-dimensional tensors Elixir lib with multi-stage... by elcritch on Friday February 19, 2021 @08:12AM (#26192756)
Ah thanks, I think HN ate my character. It’s ‘<*>’ with a asterisk in between.
Re:Nx: Multi-dimensional tensors Elixir lib with multi-stage... by elcritch on Thursday February 18, 2021 @03:35PM (#26185144)
P.S. a potential sweet spot for Elixir in the broader scientific computing community comes when you combine native numerical libraries _with_ Phoenix LiveView! Realtime scientific dashboards are amazing. There's nothing I've found that makes interactive dashboards as easy as with LiveView. Actually there are projects in R, Julia, and Python that are great but they're much more difficult to deploy and scale IMHO.
Re:Nx: Multi-dimensional tensors Elixir lib with multi-stage... by africanboy on Thursday February 18, 2021 @06:33AM (#26178627)
Elixir allows overloading the operators or creating custom ones, but only certain symbols are recognized in the parser, to add new operators the Elixir parser needs to be changed.

You can see them here https://github.com/elixir-lang/elixir/blob/master/lib/elixir...

Not all of them are defined in the language, the list of the unused infix operator includes

    \\, <-, |, ~>>, <<~, ~>, <~, <~>, <|>, <<<, >>>, |||, &&&, and ^^^
to define a custom infix operator you can use this syntax

    def left <|> right do
      IO.puts("#{left} <|> #{right}")
    end

    # usage
    left MyModule.<|> right

    # or
    import MyModule
    left <|> right
if the operator is already defined in Kernel, you can exclude it from import and use your own

    import Kernel, except: [{:+, 2}]
    import MyModule

    left + right
as you can see @ is not among them, @ it's not a binary operator and is defined as Kernel.@/1