News for hackers. Stuff that matters.
Re:What's all this fuss about Erlang? (2007) by aaron-lebo on Tuesday January 24, 2017 @03:07PM (#13474976)
I think the "more cores == faster" benefit is overstated.

Without jit Erlang is so slow that languages without concurrency support will kill it. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. Not to mention it's 10 years later and we haven't seen massively parallel architectures take off.

I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.

Re:What's all this fuss about Erlang? (2007) by stcredzero on Tuesday January 24, 2017 @03:25PM (#13475125)
I think the "more cores == faster" benefit is overstated.

I've tried to create servers for massively multiplayer games in Clojure and Go. What I've found is that it "takes some doing" to be able to efficiently use many processors in parallel. You can't just slap on a pmap call in Clojure or do the equivalent cheesy channel trick in Golang and have good utilization of all your cores in parallel.

It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so.

For what value of "mostly scale as well?" For me, that means doing a little extra design/architecting to break out your Golang server process into multiple worker processes.

it's 10 years later and we haven't seen massively parallel architectures take off.

Because there are still barriers that are a bit too large. You either wind up having to do weird things to do what amounts to efficiently passing data between cores, or you end up using a slower language that's a bit far from the computing mainstream.

Re:What's all this fuss about Erlang? (2007) by zazibar on Tuesday January 24, 2017 @03:30PM (#13475173)
Can you elaborate on how you implemented those servers? Would you still use Clojure or Go for them now? If not, what alternative do you suggest?
Re:What's all this fuss about Erlang? (2007) by stcredzero on Tuesday January 24, 2017 @04:26PM (#13475665)
My first attempt was a server for a dungeon crawler that superficially looked like a rogue-like, but was actually 12 frames/sec realtime instead of turn based. It started out on Clojure, but I afterwards ported it to Go. In this server, the world was divided into 80x24 subgrids that did most of their processing in parallel. Processing would happen in two stages: 1) local data processing, where each entity got updated in a loop, sending out updates to clients and 2) collision resolution/movement. If I had to do it over again, I would relax the grid's invariant properties, and make each 80x24 subgrid completely independent, which would eliminate the collision resolution step.

In my current server, I'm basically sharding my space-game into star systems, with free travel between star systems though "Hyperspace." There is a farm of "worker processes" that is coordinated by a master server. Any instance of a star system can be idempotently spawned when a client is attached to it. < https://www.emergencevector.com >

I think Go is a good option for writing a game server. You'd have to be pretty profligate to make the GC pause overrun one 16.66 ms tick, and if you miss a tick here and there, who cares?

Re:What's all this fuss about Erlang? (2007) by protomikron on Tuesday January 24, 2017 @08:30PM (#13477615)
> In this server, the world was divided into 80x24 subgrids that did most of their processing in parallel.

Is there any reason why you do update() in parallel? There is certainly some interesting challenges to solve there but did you have a reason to make it parallel other than intellectual curiosity (e.g. better performance)?

Afaik a game loop's update() is not the intense part in the game loop (maybe that's different for games with lots of world information update, like roguelikes), but rather render() and IO() are so I am wondering what your design thoughts are.

Re:What's all this fuss about Erlang? (2007) by stcredzero on Tuesday January 24, 2017 @08:44PM (#13477697)
Is there any reason why you do update() in parallel?

I wanted to come up with an architecture that could support an arbitrarily large world given enough resources. Up to a certain density (from 150 to 250 users per "subgrid") I was close to succeeding.

Afaik a game loop's update() is not the intense part in the game loop but rather render() and IO()

"Render" in the context of a server like this is the IO. One thing you should know, is that this game supported Conway's Life cellular automata at 12 frames/second, completely shared and multiplayer.

Re:What's all this fuss about Erlang? (2007) by sbov on Tuesday January 24, 2017 @08:16PM (#13477540)
I've always enjoyed writing MUDs (similar to roguelikes), and I found Clojure's killer feature to be immutability, rather than some of the fancier concurrency features like STM. Operations that consume the most resources in a MUD tend to be very amenable to stale, but consistent snapshots, which immutability makes pretty easy to do. My goal was more about responsiveness than throughput though.
Re:What's all this fuss about Erlang? (2007) by Vaelatern on Tuesday January 24, 2017 @04:32PM (#13475732)
Why did you leave Clojure?
Re:What's all this fuss about Erlang? (2007) by stcredzero on Tuesday January 24, 2017 @05:11PM (#13476170)
Clojure weenies on IRC. To be fair, the Go weenies aren't much better. If you like hanging out on IRC so that you can emotionally abuse newbies, ask yourself what you're doing with your life.
Re:What's all this fuss about Erlang? (2007) by Vaelatern on Tuesday January 24, 2017 @06:19PM (#13476742)
Fascinating, so it was purely a communal decision.
Re:What's all this fuss about Erlang? (2007) by zazibar on Tuesday January 24, 2017 @04:39PM (#13475800)
Interesting. Thanks for the detailed response!
Re:What's all this fuss about Erlang? (2007) by momerath on Tuesday January 24, 2017 @03:14PM (#13475029)
Unfortunately, I think you're correct.

Anecdote: I used Erlang, over the course of several years, for very-high-level bot behavior and multi-player mission control, where performance was much less important than the things Erlang provides, but I still had issues with it. Nonetheless, I felt it had given me great leverage, and I thanked Joe Armstrong profusely, when I met him at a conference. Later at the con, I asked a panel whether there were active efforts at improving performance, perhaps with a JIT. They seemed to take it as an attack, and suggested that if I need performance, I should use another language. I mostly write in Clojure now.

Re:What's all this fuss about Erlang? (2007) by im_down_w_otp on Tuesday January 24, 2017 @08:50PM (#13477727)
(facepalm) that somewhat closely mirrors my experience. It's a bit baffling.
Re:What's all this fuss about Erlang? (2007) by rdtsc on Tuesday January 24, 2017 @04:13PM (#13475517)
> I think the "more cores == faster" benefit is overstated.

I find it is not. But of course the right tool for the right job. Multiplying matrices as a single batch process is probably not going to work very well. But I saw it works exceptionally for highly concurrent workload.

This is not just "I ran a quick benchmarks on my laptop and then extrapolated by using an multiplier". Distributed systems work funny and they don't scale linearly. Sometimes a very fast, small sequential program will hit a wall and barf on itself when faced with multiple concurrent requests.

The other secret sauce is not multi-core capability but process isolation. The reason is this: a large distributed system is only as useful / practical as it is fault tolerant. It doesn't matter if you run hand tunes assembly routine and process 1M requests per second, then it crashes every day and you have hours of debugging and downtime. The average throughput or uptime will be pretty bad.

This is one key insight into what makes Erlang (and Elixir) a good tool for distributed systems - fault tolerance is built into the core of the platform.

Re:What's all this fuss about Erlang? (2007) by gamache on Tuesday January 24, 2017 @03:25PM (#13475126)
> ...It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so... > I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.

Comparing a bare-metal-but-with-GC language like Go to a miniature operating system like Erlang isn't quite fair -- in comparison, Go is missing an actor model, buffered CSP, crash supervision, hot upgrades, remote console/tracing/debugging, and built-in clustering (to name a few).

Once other languages adopt "enough" of the BEAM featureset, they'll have similar performance.

Re:What's all this fuss about Erlang? (2007) by aaron-lebo on Tuesday January 24, 2017 @03:36PM (#13475229)
Hot upgrades and remote debugging are front in center in lisp and it has excellent performance. Even so, there's no general consensus among programmers about whether hot code reloading is preferred to static binaries.

The actor model is a choice of implementation. Crash supervision stands out, but that's not necessary and is part of the Erlang culture - let it crash. It's not as important in other languages with a different underlying belief about errors.

What features mean BEAM must necessarily be slow?

Just to be clear: I really like Erlang. It obviously has different features than a lot of languages and most are aimed towards a certain goal: uptime. But if you can give just a little on that (and you might not have to at all), you can get most of those advantages with a different feature set.

I wonder out of the Erlang experiment which are the objectively better features and which are just really cool?

Re:What's all this fuss about Erlang? (2007) by davidw on Tuesday January 24, 2017 @03:10PM (#13475008)
> fold the good parts of Erlang/BEAM into them

A lot easier said than done, because there are a lot of really fundamental decisions about Erlang that make it what it is. You can't just slap on a scheduler for instance, or Erlang style processes.

Go is probably the closest thing, and it's still lacking some things like the supervision tree.

Erlang is certainly not 'the answer' in terms of computationally intensive anything, for the foreseeable future.

Re:What's all this fuss about Erlang? (2007) by lliamander on Tuesday January 24, 2017 @04:04PM (#13475423)
> Go is probably the closest thing, and it's still lacking some things like the supervision tree.

From what I've read, Go's runtime just isn't compatible with the kind fault-tolerance that Erlang provides (nor was it meant to). It lacks process monitoring, lacks process isolation, has a pause-the-world garbage collector, etc.

Not a knock against Go, just pointing out that the design goals were different.

Re:What's all this fuss about Erlang? (2007) by djKianoosh on Tuesday January 24, 2017 @03:17PM (#13475058)
so many people (including me from time to time) misunderstand "computaionally expensive" that I think Elixir/erlang/beam will serve a lot of systems/apps well for a long time until they really see a need for "intense" computation
Re:What's all this fuss about Erlang? (2007) by davidw on Tuesday January 24, 2017 @04:33PM (#13475738)
Oh, absolutely - you can do tons of things with Erlang, just as people do with Ruby, Python, PHP, Tcl, etc...

But that article is a bit hype-y.

Re:What's all this fuss about Erlang? (2007) by djKianoosh on Tuesday January 24, 2017 @05:06PM (#13476123)
well, of course it is ;) consider the source.
Re:What's all this fuss about Erlang? (2007) by zitterbewegung on Tuesday January 24, 2017 @03:10PM (#13475006)
The ways that massively parallel architectures have taken off are in GPU programming and they interact with them using the languages that they would have already done. C/C++ libraries that either use them directly or provide an interface to Python / Lua / Java etc...
Re:What's all this fuss about Erlang? (2007) by tormeh on Tuesday January 24, 2017 @03:19PM (#13475073)
The slowness of Erlang was a conscious decision. You're supposed to offload all the computationally intensive things to native code. That may not be ideal for your use case and a PITA, but it's how the language and runtime was designed. I would guess Erlang would do (if it isn't already doing) well in HPC as the thing that coordinates all the Fortran code.
Re:What's all this fuss about Erlang? (2007) by jcoffland on Tuesday January 24, 2017 @03:12PM (#13475018)
Agreed. I don't think there is a big fuss about Erlang anymore. Still it's an interesting language from an acidemic perspective if not from a practical one.
Re:What's all this fuss about Erlang? (2007) by lliamander on Tuesday January 24, 2017 @04:24PM (#13475636)
Erlang is certainly inaccessible, but I don't think academic is really the right word for it. It was born out of very practical business needs, and the reason it is still around is mainly because it has done an excellent job of serving those use cases.

What they haven't done (or to the extent that other languages have) is make the language and runtime accessible to the broader programming community. The result is that the people who (start to) use Erlang are the kind of people for whom no other tool will help them solve their problems.

Although I have not used Elixir, it seems to me that it was designed as a language that is easy enough to adopt for the wider programming community, while still leveraging the concurrency, fault-tolerance, etc. provided by BEAM and OTP. On top of that, the Elixir community has worked hard to provide the tooling and support developers typically expect these days[0].

This is not to say that Erlang should be radically altered or abandoned. I like the Erlang aesthetic and ethos, and I think the community knows the way it needs to evolve. It is just that I expect more people to come to BEAM and OTP through Elixir than Erlang, and I think it is that platform that should really be the focus.

[0] to their credit, the Erlang community has definitely improved in this area.

Re:What's all this fuss about Erlang? (2007) by tempz on Tuesday January 24, 2017 @07:11PM (#13477130)
Erlang is made for IoT.

Tens of millions of concurrent unrelated clients, embarrassingly parallelizable, scales linearly with number of cores, as these share nothing.