Two years of Elixir at The Outline

Re:Two years of Elixir at The Outline by latch on Friday October 26, 2018 @11:28PM (#18314205)

Thanks. This is really helpful. I tried to think of some specific example. I think most of what I'd generally come up with is tied to the available datastructures. We recently tested some code to essentially do a group-by in code, with various aggregation functions, and it was considerably slower in Elixir vs D and C. Very simplified, something like:

   Enum.reduce(rows, %{}, fn [user_id, hits], agg ->
     Map.update(agg, user_id, hits, &(&1 + hits))
   end)

I didn't apply any of your suggestions.

If I had another performance-like related pain point, it's throwing multiple cores at some data. I think it's somewhat common to have a large piece of data that you want to concurrently access. For example, keeping a large suffix tree in memory for an autocomplete endpoint, and letting each request concurrently access it. This isn't really a specific performance issue, but it's a performance sensitive pattern that is very antithetical to everything Erlang. ETS has limited structures, requires copying of values, and has no atomic replace (so you can't build a new version in a temp table and just rename it).

[ Parent | on HN ]

Re:Two years of Elixir at The Outline by zinclozenge on Monday October 29, 2018 @10:29AM (#18327788)

Specifically for the suffix tree in memory, you would store the tree as a constant in a module and dynamically recompile at runtime. By doing so it puts the tree in a special heap that can be read by all processes and doesn't get copied around. There's a library called 'mochiglobal' that does that for you. Discord ported it to Elixir and called it 'fastglobal' (although their implementation can be sped up by about ~25%).

[ Parent | on HN ]

Re:Two years of Elixir at The Outline by elcritch on Saturday October 27, 2018 @01:28AM (#18314486)

Tryout using Rustler and passing Refs. It’s a handy combination and Rust avoids most memory management issues one would face with NIF’s in C. I’m working on some data calculation code and think I’ll go that approach.

[ Parent | on HN ]