News for hackers. Stuff that matters.
Re:Rustler is a library for writing Erlang NIFs in safe Rust... by bryanrasmussen on Monday September 16, 2019 @03:40PM (#20987815)
Since in Erlang you should fail fast and noisily I think this Achilles heel is perhaps considered a strength.
Re:Rustler is a library for writing Erlang NIFs in safe Rust... by fabian2k on Monday September 16, 2019 @03:58PM (#20987965)
A crash in a NIF circumvents the entire supervisor architecture that makes failing fast work well in Erlang/Elixir. The idea is that a crash never pulls down the entire thing, only the parts that are directly connected. So in the original telecommunication use case, you'd happily fail a single call if something unexpected happens, but that never should take down the entire telecom switch and all other calls with it.

A bad NIF crashes the entire VM and breaks the Erlang/Elixir model of failure handling, so you really should not put code that crashes in there.

Re:Rustler is a library for writing Erlang NIFs in safe Rust... by bryanrasmussen on Monday September 16, 2019 @04:32PM (#20988277)
Thanks for pointing out the error in my thinking!
Re:Rustler is a library for writing Erlang NIFs in safe Rust... by heavenlyblue on Monday September 16, 2019 @07:20PM (#20989534)
How does Elixir handle OOMs, in such a case?

Do they fail the entire VM, too?

Re:Rustler is a library for writing Erlang NIFs in safe Rust... by toast0 on Monday September 16, 2019 @07:51PM (#20989702)
Yes mostly; when you get a failed allocation, for the most part there is nothing reasonable to do other than make a nice crash dump. Especially from the point of view of a language/toolkit.

As a operator/user, you can certainly write some code to scan for large processes, and kill them; with a fair bit of success, but that's not a great thing for the toolkit to do, it doesn't have a good way to tell what's too big or what's intended.

Re:Rustler is a library for writing Erlang NIFs in safe Rust... by ramchip on Tuesday September 17, 2019 @05:32AM (#20993242)
Nowadays in Erlang you can set a memory limit per process[1], ensuring they get killed if they try to allocate more. Of course you still have to estimate how much is reasonable which isn’t always obvious...

[1] See max_heap_size under http://erlang.org/documentation/doc-9.0/erts-9.0/doc/html/er...

Re:Rustler is a library for writing Erlang NIFs in safe Rust... by puzza007 on Monday September 16, 2019 @08:00PM (#20989766)
Yes
Re:Rustler is a library for writing Erlang NIFs in safe Rust... by walterstucco on Monday September 16, 2019 @06:23PM (#20989149)
Actually, that's not exactly what Joe Armstrong envisioned when he created Erlang, the system.

If you read his 2003[1] thesis, at page 36 he quotes a paper from Jim Gray (who worked at Tandem computers and was a Turing award winner)

    Although compiler checking and exception handling provided by programming languages are real assets, history seems to have favored the run-time checks plus the process approach to fault-containment. It has the virtue of simplicity—if a process or its processor misbehaves, stop it
and the on page 37

    The idea of “fail-fast” modules is mirrored in our guidelines for programming where we say that processes should only do when they are supposed to do according to the specification, otherwise they should crash.
and then on page 40

    Error handling is non-local.

    When we make a fault-tolerant system we need at least two physically separated computers. Using a single computer will not work, if it crashes, all is lost. The simplest fault-tolerant system we can imagine has exactly two computers, if one computer crashes, then the other computer should take over what the first computer was doing. In this simple situation even the software for fault-recovery must be non-local; the error occurs on the first machine, but is corrected by software running on the second machine.
So actually the VM crash is a signal: something's gone really bad, don't even try to recover, just let it die and let some other non-local process take over (the OS, some `forever` like script, a process on another machine...).

[1] http://erlang.org/download/armstrong_thesis_2003.pdf

Re:Rustler is a library for writing Erlang NIFs in safe Rust... by dmix on Monday September 16, 2019 @10:37PM (#20990721)
You can't put long quotes in code formatting
Re:Rustler is a library for writing Erlang NIFs in safe Rust... by sb8244 on Monday September 16, 2019 @03:52PM (#20987913)
I'd agree if it crashed the calling process, but crashing the entire VM is way too noisy of a way to go out.
Re:Rustler is a library for writing Erlang NIFs in safe Rust... by lostjohnny on Tuesday September 17, 2019 @04:54AM (#20993052)
Do you know other VMs that resist a crash in external, native, unchecked code?

A NIF is basically VM code that undergo some minor limitation and runs at full speed.

BEAM is not a sandbox and will (probably) never be.

The most important lesson Erlang teaches is that there could be no recovery from crashes from the same code that crashed.

That's why in OTP Supervisors are in the shape of a tree, each level is responsible for its children and only its children.

If you need control on some process tree, you start another supervisor to supervise it and so on.

The same goes for the VM: if you need to supervise the VM, you start an external process to check its status.

A crashed process is better than a process that misbehaves.

Non locality of error handling is the only way to be really fault tolerant.

Re:Rustler is a library for writing Erlang NIFs in safe Rust... by walterstucco on Tuesday September 17, 2019 @03:56AM (#20992717)
Is there a VM that does not crash in case of a catastrophic event?

For example: what happens if I run this code?

    public class Main {
      public static void main(String[] args) {
        Object[] o = null;

        while (true) {
            o = new Object[] {o};
        }
      }
    }
What happens if there is a bug in some JNI code that cause a segfault?

What Erlang programming style tries to teach is not to avoid crashes by being very defensive and trying to prevent every possible error condition (it's impossible), but to detect them by being fault tolerant and restart the process from a good state.