This might help: from the release notes to Erlang R20 (i.e. latest):
A new event manager to handle a subset of OS signals in Erlang
It's documented in the docs for kernel here:
http://erlang.org/documentation/doc-9.0-rc2/lib/kernel-5.3/d...
You can write your own handler, which you'll install using gen_event (or :gen_event in Elixir), but even if you don't, there's a default handler which handles 3 signals, SIGUSER1, SIGQUIT and SIGTERM.
Per the link above, SIGTERM is handled like so:
The default handler will terminate Erlang normally. This is equivalent to calling init:stop().
Also worth a look, to see how os:set_signal/2 works (this allows you to decide which signals should be ignored, handled by the default handler or handled by your own handler):
http://erlang.org/documentation/doc-9.0-rc2/lib/kernel-5.3/d...
(And a shout out to legoscia on Stack Overflow for letting us know this might be available:
So, you're welcome...and thank you for getting me to look :-)
Error messages have gotten better in Elixir 1.5 The whole ecosystem is actually getting better with each release.
For shipping to production I use 'distillery'. You create a 'release' and you can even instruct it to read some arguments from the env vars.
But yeah, more resources would definitely be welcome. Hopefully as more people start using the language, more resources will be created.
I will read more libraries; hopefully I can find some which use flow.
I have used distillery but I find myself going down a pretty big rabbit hole just to release something which in most languages is pretty simple. I will look at it again.
I based my dockerfile on https://gist.github.com/bsedat/16cb74ebc8ab0ed61ac598a129b0a...
Blogpost: https://zorbash.com/post/docker-multi-stage-elixir-distiller... (I'm not the author)
We have some on DO and some on Heroku for clients. Both work well enough.
That's my main complain about Elixir. Releasing and configuring at runtime is a huge pain. I honestly don't see any value in this whole "compile time configuration". If it's at compile time, I can put it in the code. What I call configuration is things I can change at runtime. The fact that something that is a given in any language (config via env vars) requires an extra dependency in Elixir is kind of sad.
Otherwise I love the language, the paradigm and the platform.
I don't think it's very much work - but everytime i try to build something like this I go down this path of trying to work out exactly what the architecture should be like.
Maybe it's a config application you include in extra_applications. How would you tell it the format of the file? How could you do validations?
And then maybe do an escript or similar for configuring it on the fly - something like "elixir_set <variable> <value>"
No, I believe that none of these issues are going to last very long - the community is strong, the average skill level of prolific Elixir coders and writers is pretty high, and there's tremendous traction. But currently it's the case and I'd wager that anyone raving on HN about how Elixir is the best thing since sliced bread probably didn't get super much further than a Phoenix CRUD app.
This is a fair point, even in Erlang land. There are a zillion things encouraging you to "let it crash" and far fewer going beyond that.
One thing that doesn't get mentioned often enough is a circuit-breaker like fuse: https://github.com/jlouis/fuse
This also has some more advanced topics:
Elixir 1.5 has introduced improved messages for pattern matching errors in function clauses: https://github.com/elixir-lang/elixir/blob/v1.5/CHANGELOG.md...
I will add some comments below but they are not meant to dispute your claims but to find places where we can improve.
> The log messages suck. If I want to use them in production and actually ship them to ELK or similar, the format really sucks.
Do you mean the Logger messages? Their format is customizable. Can we make this clearer?
> Getting information on what happens at compile time vs run time is hard.
Which kind of information do you need? Is it something that would fit the meta-programming guide on the website? Or maybe we are missing a more detailed compiler guide?
> How do you handle operating system signals? like when your autoscale group decides to scala down and you get a SIGTERM with some time to handle it, what do you do? wrapper scripts suck.
It was not possible cleanly until OTP 20, so it is fairly new. At least, from 19.x onwards, the VM will shutdown cleanly on SIGTERM - which means shutting down all applications and supervision trees. A lot of guides (including the Elixir guides) cover supervisor only through the fault tolerance perspective, while they also encapsulate the start-up and shutdown logic. I have opened an issue for this here: https://github.com/elixir-lang/elixir/issues/6479
> What command do I run? when do I run an app vs compiling it? What If I want command line args? Where's a good example of writing an app, which is a daemon, and using an escript to reconfigure it at runtime?
Running your code is a matter of starting your application. We don't really have a `main` procedure as an entry point. Your system is designed as a series of applications and you start them. If you are using mix, it should be a matter of `mix run --no-halt`. If you are using releases, it automatically starts the system for you. Where would you expect to find this information? Did you see `mix run` at all? Maybe in the escripts page we should point to `mix run` as it seems an `escript` is really not required in your case?
> The error messages. Holy hell. What actually caused a pattern match to not work? what is it expecting? especially a few function calls deep - like interaction between tuples and arrays and keyword lists.
Those particular error message were fixed in Elixir v1.5 (with OTP 20+). Pull request is here: https://github.com/elixir-lang/elixir/pull/6127
> Flow / Genstage and getting them into a supervision tree? what's the best practices.
Please open up an issue. If this is not clear form the documentation, it is a "bug" in the docs.
It is also worth mentioning that Ben Marx, Bruce Tate and I are working on book called "Adopting Elixir" (https://pragprog.com/book/tvmelixir/adopting-elixir). The last three chapters focus exclusively on production aspects and it answers many of the questions here (log configuration, system termination, how to run it, design, etc).
Maybe it's easier for me to walk through what I do, so I can show you where I run into problems. One of my favourite ways to learn a new language is in a REPL. I've used a lot of them in the past, and iex is pretty good. Pry (in ruby) is probably the gold standard.
Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> 1 / 0
** (ArithmeticError) bad argument in arithmetic expression
:erlang./(1, 0)
This is pretty good. Now we can move on to pry [1] pry(main)> 1 / 0
ZeroDivisionError: divided by 0
from (pry):1:in `/'
The beauty here is it's saying the same thing, but the information is slightly more precise and it tells me clearly that the function being executed came from (pry) which makes a load of sense to someone using pry. I know this is a basic example, but in elixir, as you start spawning more and more processes, deciphering the log messages gets harder.back to iex
Task.async(fn -> 1 / 0 end)
** (EXIT from #PID<0.94.0>) evaluator process exited with reason: an exception was raised:
** (ArithmeticError) bad argument in arithmetic expression
:erlang./(1, 0)
(elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:36: Task.Supervised.reply/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
07:34:45.847 [error] Task #PID<0.97.0> started from #PID<0.94.0> terminating
** (ArithmeticError) bad argument in arithmetic expression
:erlang./(1, 0)
(elixir) lib/task/supervised.ex:85: Task.Supervised.do_apply/2
(elixir) lib/task/supervised.ex:36: Task.Supervised.reply/5
(stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Function: #Function<20.99386804/0 in :erl_eval.expr/5>
Args: []
This is scary. Why's it repeated? why does one have a timestamp? how does "lib/task/supervised.ex:85: Task.Supervised.do_apply/2" get here ? I called Task.async which doesn't show in the messages?I want to reiterate at this point that it's not that bad, I think the repl is pretty good but the error messages just leave a bit to be desired. Maybe there is an article on deciphering them? But if you need that, I'll say it's a barrier.
In terms of the formatter, I need to go through my old code to remember where all those particular hurdles were, but someone has suggested a prebuilt json formatting logger which sounds good to me.
A sweet spot for Elixir (for me) is queue processing, in an autoscale group on AWS. So I have a real application in mind, and I originally knocked it up with hardly any knowledge and using genstage for back-pressure etc. in a couple of nights. This is saying a great thing about the language. But then shipping logs, deployment, debugging (if i deploy with distillery it locks up, and it has slow memory leak that makes no sense to me), dockerising, handling autoscale events have all stopped it from going to production ready.
> Where would you expect to find this information? Did you see `mix run` at all? Maybe in the escripts page we should point to `mix run` as it seems an `escript` is really not required in your case?
A guide on https://elixir-lang.org/getting-started/introduction.html would be great, to just give the different options and when you'd use them. Maybe something like "The anatomy of an elixir application" (or program because application is overloaded).
For me it is the cyclic journey through the different ways to do things. So it took a while to work out that I can't just pass in command line parameters to mix run, so the application gets them. Unless i'm wrong?
In escripts I can. And they will start the application automatically. Why would I use them? And what's the difference between them and running mix run lib/script.exs?
So then I started thinking - maybe the ideal way is to have an application running in the --no-halt way like you've described and have an escript connect to it to pass in configuration changes. But i'm just hacking different ways rather than learning from an example or best practice. I'd love config to be changeable on the fly for a daemon / application.
> Which kind of information do you need? (re: run time vs compile time)
in my limited experience, config.exs pulls in config params at compile time. I thought it would happen at run time because it's a script (exs). I know phoenix has a way of dealing with this, but i haven't gone fully down that rabbit hole yet, because I'm not even sure if config.exs is right for my requirements. If I want it to be hot loaded, I'm thinking I need a genserver with the config as its state, and some way to communicate changes to it - either from the file system or an escript.
> Those particular error message were fixed in Elixir v1.5
Thank you, it sounds like a great release
I'll open that issue, and check out your new book too because I really think there are huge gaps between
1) I can write genservers and get that
2) I know the tradeoffs and best structure for my supervision tree (especially with things like OS signals thrown in the mix)
3) I can deploy this into production
4)
5)
6) I can leverage distributed features
This means that something such as 1/0 should include the line where the error happened in IEx and in general.
Regarding the error messages, the reason why you have two reports is because there are two processes failing! The task fails, leading to the logger report, and then it causes the evaluator process to fail, causing it to be report an error as well.
Regarding configuration, it is not at compile time, but the config.exs itself is read when the escript is built. You can change it at any time though by calling Application.put_env, so there is no need for a GenServer. I know exactly where to improve the docs for this one.
Thanks for the feedback! I think I have an idea on how to improve many of those. I will do some of that and get back to you later.
That said, there are some things that suck so hard about it.
The log messages suck. If I want to use them in production and actually ship them to ELK or similar, the format really sucks.
Getting information on what happens at compile time vs run time is hard.
How do you handle operating system signals? like when your autoscale group decides to scala down and you get a SIGTERM with some time to handle it, what do you do? wrapper scripts suck.
Not many great design resources.. I mean I've got lots of books on OTP. But if i'm building a daemon which reads off a queue and talks to some databases - where are some patterns I can follow? How do I handle config? What if I want to hotload it?
What command do I run? when do I run an app vs compiling it? What If I want command line args? Where's a good example of writing an app, which is a daemon, and using an escript to reconfigure it at runtime?
The error messages. Holy hell. What actually caused a pattern match to not work? what is it expecting? especially a few function calls deep - like interaction between tuples and arrays and keyword lists.
Flow / Genstage and getting them into a supervision tree? what's the best practices.
I want to introduce it to my workplace but without some of these common patterns easy to find, it's kind of a risk to the business.