News for hackers. Stuff that matters.
Re:What's all this fuss about Erlang? by nirvana on Sunday August 05, 2012 @07:35AM (#4341198)
Erlang is a secret weapon.

Erlang is never going to be the really "hot" or "cool" language because it has an unusual syntax and that syntax is enough to keep it from becoming fashionable.

But this is a feature, actually. Because the really good engineers, they will invest the week or two (really!) to learn the language and once they do that they come to love it, and as a result, erlang has some of the best engineers working with it.

What Erlang does, you simply can't do with a library or bolt on solution, or anything involving ruby, python or the JVM. And what erlang does-- concurrency done right-- is so valuable these days that anyone solving real problems has run into it.

Erlang as been quietly winning in production and building out just about every library you could want. And because the language is so elegantly designed, for many projects its really accessible- you can dive into the code and comprehend it. (I get lost in the riak sources, because it does so much it overflows my stack, but everything else is easy.)

Erlang is not only a secret weapon, but it is a productivity multiplier.

Re:What's all this fuss about Erlang? by davidw on Sunday August 05, 2012 @08:08AM (#4341252)
> building out just about every library you could want

It's unlikely that an open source language is both "secret", and has oodles of libraries - things just don't tend to work that way.

Here's a simple anecdote: Erlang doesn't seem to have a good image manipulation library/interface to Graphics/ImageMagick. That's something most popular languages have at least one of.

That said, when you have a task that's a good fit for Erlang's sweet spot, Erlang is very good. The runtime is a very fine bit of software craftsmanship.

Re:What's all this fuss about Erlang? by quanticle on Sunday August 05, 2012 @01:38PM (#4342018)
>It's unlikely that an open source language is both "secret", and has oodles of libraries - things don't tend to work that way.

That is true, but, at the same time, irrelevant. For example, at my company we do e-mail archiving. Does Erlang have any libraries to facilitate that? No. Not even close. But Apache Lucene does. So, what we do is use the Erlang OTP Java interface [1] to allow Erlang to handle the message passing and parallelization, while Lucene handles the actual indexing and search.

The way I see it, Erlang makes it very easy to add a message passing layer on top of whatever single-threaded process you currently have. This allows you to scale your applications without having to muck about with their internals too much - you launch multiple instances of your existing single threaded code and use Erlang to handle all the messiness of message passing.

I think that is what makes Erlang such a secret weapon. The fact that you can very easily add a message passing layer to anything with Erlang greatly simplifies the job of parallelizing existing programs.

[1] http://www.erlang.org/doc/apps/jinterface/java/com/ericsson/...

Re:What's all this fuss about Erlang? by lobster_johnson on Sunday August 05, 2012 @01:22PM (#4341977)
> Erlang doesn't seem to have a good image manipulation library/interface to Graphics/ImageMagick

Ordinarily this would be a problem, but ImageMagick (since you mention it), in my experience, is something you do not want to link directly to. Despite being a quite old and (theoretically) mature project, it's quite brittle, leaky and buggy, prone to segfaulting and weird freezes. These days I just fork and exec the `identify` and `convert` libraries to do image conversion (and simple manipulations), and for graphics drawing I use other, less buggy libraries such as Cairo.

Re:What's all this fuss about Erlang? by rdtsc on Sunday August 05, 2012 @08:55AM (#4341316)
> It's unlikely that an open source language is both "secret", and has oodles of libraries - things just don't tend to work that way.

It really depends on the history and the typical domain that language is used in. Yes, Erlang doesn't have good image manipulation library, but it has built in support for a multi-node distributed system setup. Applications with automatic failover running on 2 nodes. Also a distributed database and so on. These are not trivial things to just whip up on other languages, but Erlang has them in the standard library.

It has also been used for telecom applications for a long time before it became open source, and thus it accumulated a decent size library over the years, which it might not have had it been a "secret" language started from scratch by a PhD student 5 years ago...

Re:What's all this fuss about Erlang? by maximveksler on Sunday August 05, 2012 @08:25AM (#4341272)
Erlang is a powerful language, IMHO it stands out because of 3 features:

- Actor Model

- Pattern Matching

- Hot code replace

Yet HotSpot is more performant VM then erlang's VM is. Now, if those features could be implemented on top of the JVM it would (to me at least) render erlang to be irrelevant.

So let's examine what is currently known.

- Actor Model: An example of a massive implementing effort of Actor model would be scala's Akka framework http://akka.io/

- Hot code replace: I don't know the answer to whether HotSpot lame code swapping implementation can be fixed to act like erlang does it. http://scala-programming-language.1934581.n4.nabble.com/how-...

- Pattern Matching: As for pattern matching, to my understanding this can be implemented on top of the JVM, if one wishes to. http://news.ycombinator.com/item?id=2784086

Re:What's all this fuss about Erlang? by pka on Sunday August 05, 2012 @08:56AM (#4341322)
You're forgetting about OTP, Erlang's battle tested framework that covers about any edge case conceivable in a concurrent world. By themselves, you could probably produce hacky actors, pattern matching and hod code swap implementations in most dynamic languages (and on VMs like NET or JVM) nowadays, but it's the whole package that has remained unmatched so far.

Not to mention Erlang's live console and logging capabilities which are indispensable when it comes to diagnosing crashed services or inspecting live systems and gathering stats. The fact that Erlang's data structures are immutable allows OTP to give you a very detailed report containing the exact state before the crash and the message that led to it, while in Java and most other languages your best bet is just an exception trace - and then you would have to guess what exactly went wrong because you don't have the system state anymore and can't recreate it easily for that matter. You can imagine that this makes debugging an Erlang application a joy - at least as joyous as it can get.

One other important aspect of OTP are supervisor hierarchies - Erlang is built around the idea that if something can fail, it will - so systems should be designed in such a manner that failure in one component doesn't tear down the whole system. Services are carefully subdivided into autonomous submodules, and if a module crashes OTP will attempt to restart it and continue its operation if possible. Imagine a png conversion submodule crashing because of a faulty png header - in a Java system this would most probably throw an IndexOutOfBoundsException and down comes your whole backend. Erlang/OTP would just note that the converter has crashed and then restart it. It can do that because the converter doesn't affect or depend on some interconnected, global, "complected" state and thus can be easily started and stopped at will.

Of course, Erlang's solution to concurrency is no silver bullet - you can still create deadlocks if you aren't careful and you can still create global state by abusing named processes, but it makes a lot of things a lot, lot easier.

edit: one more thing - Erlang's hot code swap support is really one of the most beautiful things I've seen when it comes language features. Now, swapping code by itself isn't such a big problem - you can do this in C as well by just reloading a DLL or in Java by un/reloading a specific class - but the tricky part is about swapping state, and this is where most imperative languages fail hard.

Said otherwise, when you update your service, you not only want to update your code but also your data structures. For example, if you have a class/struct like this:

    struct person {
        string name;
        int age;
    };
and would like to add an additional field `ageString` to it, for whatever purposes, how would you go about doing this in a more conventional language? You would have to track all usages of `person` somehow, and then replace the in-memory representation with the new version while making sure that no thread is reading or writing data, then swap the code while being careful that no new code is reading old data or vice versa. In fact, in most systems I've worked on such a thing would be almost impossible.

Erlang's processes and immutable data structures allow you to do just that - and as painless as possible. You provide a function that converts old state to new state (in this case containing the field `ageString`), the new code which operates upon the updated data structures and let Erlang do the rest. All messages that come in while the swap is taking place will be queued up and delivered later.

Moreover, if the update introduces a bug, you can as easily downgrade as you did upgrade. And all this while your service is up and running!

Re:What's all this fuss about Erlang? by blaforge on Sunday August 12, 2012 @12:31AM (#4371844)
Exceptions can be quite problematic. This is why JActor has taken such pains in its exception handler. One particularly nice feature is that, since JActor supports 2-way messages, the default exception handler passes exceptions to the active exception handler of requesting actor. https://github.com/laforge49/JActor
Re:What's all this fuss about Erlang? by malkia on Sunday August 05, 2012 @04:43PM (#4342583)
Re:What's all this fuss about Erlang? by theatrus2 on Sunday August 05, 2012 @12:17PM (#4341805)
Scala and Akka solve many of these problems on the JVM, with a strongly typed language to boot.

I love Erlang, but it's not all roses either.

Re:What's all this fuss about Erlang? by rdtsc on Sunday August 05, 2012 @09:04AM (#4341333)
The main power of Erlang comes from the ability to build fault tolerant and highly available systems. Fault tolerance was the main motivator behind Erlang's existence, not any of the thing you mention. Two of those things (actor model and hot code replace) fall out of that one requirement. Fault tolerance needs process isolation, that leads to actor model. Minimization of shared state leads to a immutable data structures + a functional approach. High availability and tolerance for upgrade downtime leads to hot code replace.

And fault tolerance is exactly what JVM doesn't have. It shares a common garbage collector and there are other things.

That is the main problem with those who claim "oh here is a faster Erlang library in language X", yes, one can copy some of the features into language X but unless full process isolation is copied, it is not really an Erlang replacement library/lanuage.

Re:What's all this fuss about Erlang? by asabil on Sunday August 05, 2012 @09:43AM (#4341403)
You can add the soft realtime characteristics of the VM to the mix. That's not something you can just add to the JVM as a library either.
Re:What's all this fuss about Erlang? by billrobertson42 on Sunday August 05, 2012 @07:06PM (#4342963)
You're either realtime or you're not. If you have hard and fast time requirements to perform certain actions then you're real time. (e.g. 3 ms. to decide whether or not to turn the cruise missile or it hits a hospital). There is no 'soft' realtime.
Re:What's all this fuss about Erlang? by rdtsc on Sunday August 05, 2012 @09:34PM (#4343336)
There is soft realtime. You don't have to have hard and fast time requirements, or nothing, you can have more relaxed statistically defined requirements. That is soft realtime.

Systems are not divided into cruise missile control systems and everything else. There is a continuum in between. A phone system where the calls need to routed in the 100 milliseconds or less 99.999% of the time is acceptable for example, while one where no such guarantees can be provided at all is not acceptable. It is not a realtime systems but it is pretty close, so it would be called soft realtime.

Re:What's all this fuss about Erlang? by lazyjones on Sunday August 05, 2012 @10:15AM (#4341473)
Erlang does not have a common runtime for its processes? What happens to the isolated Erlang processes when the VM fails to allocate more memory from the OS? Does the whole VM crash, or just the process that caused the memory allocation attempt?
Re:What's all this fuss about Erlang? by vonkow on Sunday August 05, 2012 @12:55PM (#4341904)
That is one of the few cases that will take down the whole VM (at least, as of a few versions ago). Due to Erlang's concept of process isolation, there's not really a sensible course of action for the VM to take if it runs out of memory. Maybe just killing the individual process or running a GC pass on every process and trying again might be a better approach than hosing the whole runtime, maybe they've fixed it. Either way, the frequency with which this happens in production Erlang apps seems to be low enough that it's essentially a non-issue for most of the Erlang community.
Re:What's all this fuss about Erlang? by rdtsc on Sunday August 05, 2012 @02:34PM (#4342175)
> What happens to the isolated Erlang processes when the VM fails to allocate more memory from the OS?

It suspends your process, then logs into newegg.com and buys more memory sticks for your sever. You know it is just a regular VM and doesn't have magic in it. If you use up all the memory on your server you are screwed probably.

You could set up a monitor to see which erlang process is eating memory and what rate and then decide what to do, kill the process, restart the whole node, or report someplace.

EDIT: see http://www.erlang.org/doc/man/memsup.html

Re:What's all this fuss about Erlang? by blaforge on Sunday August 12, 2012 @12:35AM (#4371848)
Take a look at JActor, a high-throughput Java actor framework. It gains its speed by operating mostly synchronously and delivers between 80 and 200 million messages per second, depending on the mode of delivery. https://github.com/laforge49/JActor
Re:What's all this fuss about Erlang? by edwinnathaniel on Sunday August 05, 2012 @11:34AM (#4341679)
Hotswapping: JRebel.
Re:What's all this fuss about Erlang? by ryanong on Sunday August 05, 2012 @08:08AM (#4341251)
Jose Valim made an awesome language for the erlang vm called elixir

http://elixir-lang.org/

I have been playing around with it and it simplifies a lot of the things that bother me about erlang. It breaks some functional conventions for code readability such as rewriting the same variable

Re:What's all this fuss about Erlang? by sandGorgon on Sunday August 05, 2012 @10:37AM (#4341522)
Thank you for this! Ruby syntax on top of the erlang VM - I wonder why this is not more popular.
Re:What's all this fuss about Erlang? by rubyrescue on Sunday August 05, 2012 @12:52PM (#4341897)
It's really, really new. We've been playing w/it a lot @ Inaka (and we do about 30% Erlang consulting projects). It will catch on, i'm sure.
Re:What's all this fuss about Erlang? by sandGorgon on Sunday August 05, 2012 @02:58PM (#4342257)
could you tell us what you are typically using it for ? Coming from a ruby-centric world, I am having difficulty visualizing how ruby and erlang could co-exist. what cant you get from Eventmachine ?
Re:What's all this fuss about Erlang? by dsrguru on Sunday August 05, 2012 @02:08PM (#4342099)
"What Erlang does, you simply can't do with ... anything involving ... the JVM."

As a matter of fact, Clojure runs atop the JVM and has a concurrency model that parallelizes even better than Erlang's, although Erlang's model of concurrency is presumably more fault tolerant.

Re:What's all this fuss about Erlang? by rcb on Sunday August 05, 2012 @04:18PM (#4342503)
http://blog.getprismatic.com/blog/2012/4/5/software-engineer...

"While we make heavy use of the core of Clojure, we don't use its concurrency primitives (atoms, refs, STM, etc.) because a function like pmap doesn't have enough fine grained control for our needs."

Re:What's all this fuss about Erlang? by bascule on Sunday August 05, 2012 @02:20PM (#4342132)
Not sure what you mean by "parallelizes better". However Erjang and Kilim on the JVM have managed to outperform the Erlang VM (thanks to a shared heap they provide zero-copy messaging unlike the Erlang VM) and with InvokeDynamic Erjang should be getting a nice performance boost as soon as they integrate it.
Re:What's all this fuss about Erlang? by gcr on Sunday August 05, 2012 @03:28PM (#4342344)
How so?
Re:What's all this fuss about Erlang? by dsrguru on Sunday August 05, 2012 @04:49PM (#4342606)
By conceptually separating synchronization from mutual exclusion and abstracting concurrency into separate models (atoms, agents, refs) that correspond to separate use cases (synchronous and independent mutable access across threads, asynchronous and independent access, and synchronous and coordinated access). It's higher level than using message passing for all types of concurrency and, therefore, more expressive.

When the goal of concurrency is parallelization, I'd want the most abstract and powerful tool for the job, which for me means Clojure over Erlang. When the goal of concurrency is reliability, I imagine I'd want separate lightweight processes for fault tolerance, which is where Erlang seems to excel.