News for hackers. Stuff that matters.
Re:Why Elixir (2014) by anko on Friday August 18, 2017 @04:01AM (#15044391)
I really am in awe of Elixir. I think it's an amazing language, and it introduced me to Erlang and OTP. I've come from using scala actors, and I can do more in way less time, less memory, and similar performance if I use Elixir.

That said, there are some things that suck so hard about it.

The log messages suck. If I want to use them in production and actually ship them to ELK or similar, the format really sucks.

Getting information on what happens at compile time vs run time is hard.

How do you handle operating system signals? like when your autoscale group decides to scala down and you get a SIGTERM with some time to handle it, what do you do? wrapper scripts suck.

Not many great design resources.. I mean I've got lots of books on OTP. But if i'm building a daemon which reads off a queue and talks to some databases - where are some patterns I can follow? How do I handle config? What if I want to hotload it?

What command do I run? when do I run an app vs compiling it? What If I want command line args? Where's a good example of writing an app, which is a daemon, and using an escript to reconfigure it at runtime?

The error messages. Holy hell. What actually caused a pattern match to not work? what is it expecting? especially a few function calls deep - like interaction between tuples and arrays and keyword lists.

Flow / Genstage and getting them into a supervision tree? what's the best practices.

I want to introduce it to my workplace but without some of these common patterns easy to find, it's kind of a risk to the business.

Re:Why Elixir (2014) by qohen on Friday August 18, 2017 @09:28AM (#15045878)
How do you handle operating system signals? like when your autoscale group decides to scala down and you get a SIGTERM with some time to handle it, what do you do?

This might help: from the release notes to Erlang R20 (i.e. latest):

A new event manager to handle a subset of OS signals in Erlang

It's documented in the docs for kernel here:

http://erlang.org/documentation/doc-9.0-rc2/lib/kernel-5.3/d...

You can write your own handler, which you'll install using gen_event (or :gen_event in Elixir), but even if you don't, there's a default handler which handles 3 signals, SIGUSER1, SIGQUIT and SIGTERM.

Per the link above, SIGTERM is handled like so:

The default handler will terminate Erlang normally. This is equivalent to calling init:stop().

Also worth a look, to see how os:set_signal/2 works (this allows you to decide which signals should be ignored, handled by the default handler or handled by your own handler):

http://erlang.org/documentation/doc-9.0-rc2/lib/kernel-5.3/d...

(And a shout out to legoscia on Stack Overflow for letting us know this might be available:

https://stackoverflow.com/a/41872533 )

Re:Why Elixir (2014) by anko on Friday August 18, 2017 @05:25PM (#15049839)
This is brilliant, I can't believe I missed it! Thank you
Re:Why Elixir (2014) by qohen on Friday August 18, 2017 @08:02PM (#15050696)
If you hadn't mentioned it, I wouldn't have gone researching it and then finding legocia's 2017 comment to a 2010 StackOverflow thread where he says that perhaps a pull request involving an "event manager for signals" might make it into R20.

So, you're welcome...and thank you for getting me to look :-)

Re:Why Elixir (2014) by plainOldText on Friday August 18, 2017 @04:36AM (#15044527)
I know it's probably not the answer you're looking for, but you can learn a lot by just reading the code of popular Elixir libraries, such as Ecto, Phoenix, etc.

Error messages have gotten better in Elixir 1.5 The whole ecosystem is actually getting better with each release.

For shipping to production I use 'distillery'. You create a 'release' and you can even instruct it to read some arguments from the env vars.

But yeah, more resources would definitely be welcome. Hopefully as more people start using the language, more resources will be created.

Re:Why Elixir (2014) by anko on Friday August 18, 2017 @05:08AM (#15044657)
I appreciate the response, at any rate.

I will read more libraries; hopefully I can find some which use flow.

I have used distillery but I find myself going down a pretty big rabbit hole just to release something which in most languages is pretty simple. I will look at it again.

Re:Why Elixir (2014) by andruby on Friday August 18, 2017 @05:52AM (#15044839)
If it's any help: I use docker multistage to build a barebones alpine docker image. It uses distillery.

I based my dockerfile on https://gist.github.com/bsedat/16cb74ebc8ab0ed61ac598a129b0a...

Blogpost: https://zorbash.com/post/docker-multi-stage-elixir-distiller... (I'm not the author)

Re:Why Elixir (2014) by thejosh on Friday August 18, 2017 @06:57AM (#15045099)
I just got the same setup this week. Docker multistage builds are amazing and make builds super small and awesome. You can ship releases without having to install Erlang or elixir on the imag, and just use what distillery has. Distillery really is amazing.
Re:Why Elixir (2014) by tmat on Friday August 18, 2017 @09:01AM (#15045680)
Releases and deployments with Elixir are still the area that needs most work, but it's nothing that would ever prevent me from using it. I have several in production and once we figured it out, it was easy peasy from there on.

We have some on DO and some on Heroku for clients. Both work well enough.

Re:Why Elixir (2014) by dudul on Friday August 18, 2017 @08:22AM (#15045472)
> You create a 'release' and you can even instruct it to read some arguments from the env vars.

That's my main complain about Elixir. Releasing and configuring at runtime is a huge pain. I honestly don't see any value in this whole "compile time configuration". If it's at compile time, I can put it in the code. What I call configuration is things I can change at runtime. The fact that something that is a given in any language (config via env vars) requires an extra dependency in Elixir is kind of sad.

Otherwise I love the language, the paradigm and the platform.

Re:Why Elixir (2014) by anko on Saturday August 19, 2017 @03:44AM (#15052215)
Yep, I think there should be some included Config behaviour module which handles the serialising / deserialising of a config file, and genservers can register themselves against it, and when the config changes the registered genservers get terminated and restarted gracefully.

I don't think it's very much work - but everytime i try to build something like this I go down this path of trying to work out exactly what the architecture should be like.

Maybe it's a config application you include in extra_applications. How would you tell it the format of the file? How could you do validations?

And then maybe do an escript or similar for configuring it on the fly - something like "elixir_set <variable> <value>"

Re:Why Elixir (2014) by plainOldText on Friday August 18, 2017 @01:57PM (#15048342)
I think I fail to see your point. You can fetch any environment variable – e.g. `System.get_env("SHELL")` – and do whatever with it. But distillery has its own way of defining configurations. You can still mix and match.
Re:Why Elixir (2014) by skrebbel on Friday August 18, 2017 @10:43AM (#15046596)
I think it's good that this is mentioned for a change. Elixir is great and HN has rightly been raving about it lately, but 95% of the Elixir resources out there are "getting started" level tutorials and blog posts. The language is too young for a lot of patterns and approaches for slightly more complex problems to have settled in people's collective minds. Plus, unfortunately there is a strong culture of reinventing the wheel and not looking a lot at things coming out of much more mature Erlang community (with the notable exception of the OTP, but that's a builtin). For example, Webmachine is amazing and totally ignored in Elixir land - unrightfully so in my opinion.

No, I believe that none of these issues are going to last very long - the community is strong, the average skill level of prolific Elixir coders and writers is pretty high, and there's tremendous traction. But currently it's the case and I'd wager that anyone raving on HN about how Elixir is the best thing since sliced bread probably didn't get super much further than a Phoenix CRUD app.

Re:Why Elixir (2014) by davidw on Friday August 18, 2017 @11:54AM (#15047277)
> I mean I've got lots of books on OTP. But if i'm building a daemon which reads off a queue and talks to some databases - where are some patterns I can follow?

This is a fair point, even in Erlang land. There are a zillion things encouraging you to "let it crash" and far fewer going beyond that.

One thing that doesn't get mentioned often enough is a circuit-breaker like fuse: https://github.com/jlouis/fuse

This also has some more advanced topics:

https://www.erlang-in-anger.com/

Re:Why Elixir (2014) by agd on Friday August 18, 2017 @09:08AM (#15045739)
> The error messages. Holy hell. What actually caused a pattern match to not work? what is it expecting?

Elixir 1.5 has introduced improved messages for pattern matching errors in function clauses: https://github.com/elixir-lang/elixir/blob/v1.5/CHANGELOG.md...

Re:Why Elixir (2014) by anko on Friday August 18, 2017 @07:02PM (#15050425)
This is really great. Maybe it's been working for me recently and I didn't realise.
Re:Why Elixir (2014) by crymer11 on Friday August 18, 2017 @08:33AM (#15045534)
We use https://github.com/rentpath/ex_json_logger to make logs nicer for consumption by ELK and the like.
Re:Why Elixir (2014) by anko on Friday August 18, 2017 @07:02PM (#15050419)
pro tip, thanks
Re:Why Elixir (2014) by josevalim on Friday August 18, 2017 @04:37PM (#15049528)
This is very good feedback and show flaws in areas that could be improved. Thank you!

I will add some comments below but they are not meant to dispute your claims but to find places where we can improve.

> The log messages suck. If I want to use them in production and actually ship them to ELK or similar, the format really sucks.

Do you mean the Logger messages? Their format is customizable. Can we make this clearer?

> Getting information on what happens at compile time vs run time is hard.

Which kind of information do you need? Is it something that would fit the meta-programming guide on the website? Or maybe we are missing a more detailed compiler guide?

> How do you handle operating system signals? like when your autoscale group decides to scala down and you get a SIGTERM with some time to handle it, what do you do? wrapper scripts suck.

It was not possible cleanly until OTP 20, so it is fairly new. At least, from 19.x onwards, the VM will shutdown cleanly on SIGTERM - which means shutting down all applications and supervision trees. A lot of guides (including the Elixir guides) cover supervisor only through the fault tolerance perspective, while they also encapsulate the start-up and shutdown logic. I have opened an issue for this here: https://github.com/elixir-lang/elixir/issues/6479

> What command do I run? when do I run an app vs compiling it? What If I want command line args? Where's a good example of writing an app, which is a daemon, and using an escript to reconfigure it at runtime?

Running your code is a matter of starting your application. We don't really have a `main` procedure as an entry point. Your system is designed as a series of applications and you start them. If you are using mix, it should be a matter of `mix run --no-halt`. If you are using releases, it automatically starts the system for you. Where would you expect to find this information? Did you see `mix run` at all? Maybe in the escripts page we should point to `mix run` as it seems an `escript` is really not required in your case?

> The error messages. Holy hell. What actually caused a pattern match to not work? what is it expecting? especially a few function calls deep - like interaction between tuples and arrays and keyword lists.

Those particular error message were fixed in Elixir v1.5 (with OTP 20+). Pull request is here: https://github.com/elixir-lang/elixir/pull/6127

> Flow / Genstage and getting them into a supervision tree? what's the best practices.

Please open up an issue. If this is not clear form the documentation, it is a "bug" in the docs.

It is also worth mentioning that Ben Marx, Bruce Tate and I are working on book called "Adopting Elixir" (https://pragprog.com/book/tvmelixir/adopting-elixir). The last three chapters focus exclusively on production aspects and it answers many of the questions here (log configuration, system termination, how to run it, design, etc).

Re:Why Elixir (2014) by anko on Friday August 18, 2017 @06:28PM (#15050246)
José, thanks for taking the time to respond to my somewhat rambling comment. To be honest, if I had a clear idea on how to solve some of these things I would have done it - but i'm still a novice at the otp ecosystem in general.

Maybe it's easier for me to walk through what I do, so I can show you where I run into problems. One of my favourite ways to learn a new language is in a REPL. I've used a lot of them in the past, and iex is pretty good. Pry (in ruby) is probably the gold standard.

  Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help)
  iex(1)> 1 / 0
  ** (ArithmeticError) bad argument in arithmetic expression
      :erlang./(1, 0)
This is pretty good. Now we can move on to pry

  [1] pry(main)> 1 / 0
  ZeroDivisionError: divided by 0
  from (pry):1:in `/'
The beauty here is it's saying the same thing, but the information is slightly more precise and it tells me clearly that the function being executed came from (pry) which makes a load of sense to someone using pry. I know this is a basic example, but in elixir, as you start spawning more and more processes, deciphering the log messages gets harder.

back to iex

  Task.async(fn -> 1 / 0 end)
  ** (EXIT from #PID<0.94.0>) evaluator process exited with reason: an exception was raised:
      ** (ArithmeticError) bad argument in arithmetic expression
          :erlang./(1, 0)
          (elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
          (elixir) lib/task/supervised.ex:36: Task.Supervised.reply/5
          (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
  
  
  07:34:45.847 [error] Task #PID<0.97.0> started from #PID<0.94.0> terminating
  ** (ArithmeticError) bad argument in arithmetic expression
      :erlang./(1, 0)
      (elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
      (elixir) lib/task/supervised.ex:36: Task.Supervised.reply/5
      (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
  Function: #Function<20.99386804/0 in :erl_eval.expr/5>
      Args: []
This is scary. Why's it repeated? why does one have a timestamp? how does "lib/task/supervised.ex:85: Task.Supervised.do_apply/2" get here ? I called Task.async which doesn't show in the messages?

I want to reiterate at this point that it's not that bad, I think the repl is pretty good but the error messages just leave a bit to be desired. Maybe there is an article on deciphering them? But if you need that, I'll say it's a barrier.

In terms of the formatter, I need to go through my old code to remember where all those particular hurdles were, but someone has suggested a prebuilt json formatting logger which sounds good to me.

A sweet spot for Elixir (for me) is queue processing, in an autoscale group on AWS. So I have a real application in mind, and I originally knocked it up with hardly any knowledge and using genstage for back-pressure etc. in a couple of nights. This is saying a great thing about the language. But then shipping logs, deployment, debugging (if i deploy with distillery it locks up, and it has slow memory leak that makes no sense to me), dockerising, handling autoscale events have all stopped it from going to production ready.

> Where would you expect to find this information? Did you see `mix run` at all? Maybe in the escripts page we should point to `mix run` as it seems an `escript` is really not required in your case?

A guide on https://elixir-lang.org/getting-started/introduction.html would be great, to just give the different options and when you'd use them. Maybe something like "The anatomy of an elixir application" (or program because application is overloaded).

For me it is the cyclic journey through the different ways to do things. So it took a while to work out that I can't just pass in command line parameters to mix run, so the application gets them. Unless i'm wrong?

In escripts I can. And they will start the application automatically. Why would I use them? And what's the difference between them and running mix run lib/script.exs?

So then I started thinking - maybe the ideal way is to have an application running in the --no-halt way like you've described and have an escript connect to it to pass in configuration changes. But i'm just hacking different ways rather than learning from an example or best practice. I'd love config to be changeable on the fly for a daemon / application.

> Which kind of information do you need? (re: run time vs compile time)

in my limited experience, config.exs pulls in config params at compile time. I thought it would happen at run time because it's a script (exs). I know phoenix has a way of dealing with this, but i haven't gone fully down that rabbit hole yet, because I'm not even sure if config.exs is right for my requirements. If I want it to be hot loaded, I'm thinking I need a genserver with the config as its state, and some way to communicate changes to it - either from the file system or an escript.

> Those particular error message were fixed in Elixir v1.5

Thank you, it sounds like a great release

I'll open that issue, and check out your new book too because I really think there are huge gaps between

1) I can write genservers and get that

2) I know the tradeoffs and best structure for my supervision tree (especially with things like OS signals thrown in the mix)

3) I can deploy this into production

4)

5)

6) I can leverage distributed features

Re:Why Elixir (2014) by josevalim on Saturday August 19, 2017 @02:41AM (#15052029)
Stacktrace entries for emulator calls are coming on Erlang 21: https://github.com/erlang/otp/pull/1478

This means that something such as 1/0 should include the line where the error happened in IEx and in general.

Regarding the error messages, the reason why you have two reports is because there are two processes failing! The task fails, leading to the logger report, and then it causes the evaluator process to fail, causing it to be report an error as well.

Regarding configuration, it is not at compile time, but the config.exs itself is read when the escript is built. You can change it at any time though by calling Application.put_env, so there is no need for a GenServer. I know exactly where to improve the docs for this one.

Thanks for the feedback! I think I have an idea on how to improve many of those. I will do some of that and get back to you later.

Re:Why Elixir (2014) by abledon on Friday August 18, 2017 @10:08AM (#15046292)
Do you play an instrument?