News for hackers. Stuff that matters.
Re:Why scaling and parallelism remain hard even with new too... by rdtsc on Wednesday August 10, 2016 @01:36PM (#12263465)
That's why I like Erlang and Elixir. They were built to handle concurrency down to the core. Currently only languages on BEAM VM provide a set of mature, built-in fault tolerance features (code reloading, isolated concurrency units -- processes, immutable data, sending messages between processes instead of acquiring lock).

I often see some frameworks which claim implement Erlang in "language $x" by adding a queue to a thread. But that is still very much behind what Erlang does, because it is missing these other components, namely fault tolerance.

Sure you can spawn OS processes, or spin up multiple machines/containers. But that is not built-in, so have to manage the additional stack for that. Java has code reloading, but it is not quite the same and so on.

And this is not just talk in generalities, these properties translate directly to benefits and profits -- faster development time, less ops overhead (some parts of the backend can crash and restart, without having to wake everyone up), less code to maintain and dependencies to manage.

Just the other day had a typical distributed systems problem -- didn't add backpressure and so messages were piling up in receiver mailbox. The simplest solution I tested was just switch a gen_server:cast to a gen_server:call. It was a 2-3 line change. Hotpatched on a test cluster and problem was fixed in a few minutes. Ultimately I did something else, but the point is using a language which was built in for concurrency is that it was just a couple of lines of code. Had that been a custom RPC solution, with some serialization and some socket code, it would have taken a lot more to write, test and deploy. All that adds up quickly and can make or break the project.

Re:Why scaling and parallelism remain hard even with new too... by heavenlyhash on Wednesday August 10, 2016 @04:11PM (#12264474)
The biggest question I have about this: Is it really a good idea to make local actors and supervisor trees indistinguishable from remote ones?

Reusability is good.

Abstractions that are so "good" that they obscure potentially order-of-mag performance differences -- I'm not sold. Whether an actor is in-memory with me or across a TCP socket is order-of-mag, and may make the difference between whether my program satisfies is service goals, or fails them abjectly. Especially if that randomized latency happens a number of times, or in unpredictable places.

Error handling paths that have to account for the Two-Generals problem being indistinguishable from local errors I can be confident I'll hear about -- I'm not sold on that, either.

I'd be interested to hear rebuttals to those from folks who have worked deeply with OTP. Is it just... not that bad, in practice? Or are there parts of the abstraction that specifically help counter these concerns?

Re:Why scaling and parallelism remain hard even with new too... by pyotrgalois on Wednesday August 10, 2016 @04:33PM (#12264575)
In some way this article by one of Erlang creators is related to your comment: https://armstrongonsoftware.blogspot.com.ar/2008/05/road-we-...

> The fundamental problem with taking a remote operation and wrapping it up so that it looks like a local operation is that the failure modes of local and remote operations are completely different.

Re:Why scaling and parallelism remain hard even with new too... by derefr on Wednesday August 10, 2016 @07:52PM (#12265488)
I think of Erlang less as making remote operations masquerade as local ones, and more as forcing you to assume that all local code has the potential failure-modes of remote code. Which, often (for weird and roundabout reasons) it does.
Re:Why scaling and parallelism remain hard even with new too... by im_down_w_otp on Thursday August 11, 2016 @02:54AM (#12266645)
This.
Re:Why scaling and parallelism remain hard even with new too... by jeffdavis on Thursday August 11, 2016 @01:38AM (#12266447)
Very insightful!
Re:Why scaling and parallelism remain hard even with new too... by qznc on Wednesday August 10, 2016 @05:23PM (#12264846)
Armstrong clearly sees the problem. I understand his solution as: Erlang makes local and remote actors indistinguishable, but you should build your own custom wrapper for remote calls.

This seems fishy to me. You build low-level boilerplate over a high-level foundation? To clarify, I consider implementing the timeout as "low-level" and a mechanism like "link(remote_process)" as "high-level".

Re:Why scaling and parallelism remain hard even with new too... by pyotrgalois on Thursday August 11, 2016 @10:29AM (#12268492)
Could you please explain why do you think a timeout would be low level?
Re:Why scaling and parallelism remain hard even with new too... by abecedarius on Wednesday August 10, 2016 @05:53PM (#12264987)
You want two kinds of abstractions: ones that can only run locally, and ones that may be remote. For the remote kind it should be easy and transparent to run them locally.

In Erlang these are function calls vs. message sending. Some other languages like E also make that kind of distinction. I believe the bad rep of network transparency came from experience trying to cross the line in the other direction: trying to pretend remote services are local.

Re:Why scaling and parallelism remain hard even with new too... by toast0 on Thursday August 11, 2016 @03:07AM (#12266684)
> The biggest question I have about this: Is it really a good idea to make local actors and supervisor trees indistinguishable from remote ones?

The erlang code to send a message to a remote pid and a local pid is the same, but that doesn't make them indistinguishable; you can check if the pid is local or not in code. In my experience, I have some expectations of where the thing I'm messaging is going to be when I write the code, and I can act accordingly: better error handling when I expect to talk to a remote node since I can't be sure I'll be able to reach it; whereas if I expect to reach a local pid, I'm more likely to match against success and let it crash if something weird happened.

Re:Why scaling and parallelism remain hard even with new too... by catnaroek on Wednesday August 10, 2016 @11:44PM (#12266210)
> They were built to handle concurrency down to the core.

And they weren't built to handle data parallelism at all.

Re:Why scaling and parallelism remain hard even with new too... by rdtsc on Thursday August 11, 2016 @04:13AM (#12266836)
Not sure I know what you mean by "data parallelism". I wrote code not too long ago which had multiple processes reference the same binary blob. I have enough CPUs to ensure some processes were actually accessing it in parallel. In other case, used an ETS table to have multiple processes read and write data to and from it.

Or maybe to put it another way, if you think I am not already doing data parallelism, sell me on why I need data parallelism.

Re:Why scaling and parallelism remain hard even with new too... by catnaroek on Thursday August 11, 2016 @06:06AM (#12267102)
> reference the same binary blob

Is there some way to share data whose structure is actually understood and checked (statically or dynamically) by the language?

> to have multiple processes read and write data to and from it.

How did you coordinate those reads and writes? A writer not having exclusive access to what it's writing to, at the time of the writing... sounds... very bad.

> sell me on why I need data parallelism.

Just to name one example: machine learning. If you're feeding real-time data to a ML algorithm, chances are you'll have to perform multiple independent calculations that use the same data as input. To speed up those calculations, you need to perform them in parallel.

Re:Why scaling and parallelism remain hard even with new too... by di4na on Thursday August 11, 2016 @04:31AM (#12266883)
Nothing is. There was an interesting discussion between Simon Peyton Jones and Joe Armstrong on that. https://www.infoq.com/interviews/armstrong-peyton-jones-erla...

In general, the thing is... parallelism is hard. If not impossible to do right from automated tools.

Re:Why scaling and parallelism remain hard even with new too... by catnaroek on Thursday August 11, 2016 @06:40AM (#12267192)
First we have to make it safe: make it inexpressible to do parallelism wrong. Only then can we discuss automating it. There is no value whatsoever in automating the wrong thing.
Re:Why scaling and parallelism remain hard even with new too... by di4na on Friday August 12, 2016 @07:10AM (#12274638)
What do you mean by that. Because doing parallelism wrong is easy : do it. Parrallelism mean doing the same thing at the same time. The only way to not do it wrong is to not do it.
Re:Why scaling and parallelism remain hard even with new too... by catnaroek on Friday August 12, 2016 @08:00AM (#12274874)
> Parrallelism mean doing the same thing at the same time.

Yeah, but how to do it without making the program state go kaboom?

> What do you mean by that.

“Make it impossible to write to a data structure without (at least temporary) exclusive access to it.” At the risk of oversimplifying, there are basically three ways to do it:

(0) Use data structures that are immutable after they are first written to. Pros: very easy to understand and use. Cons: limited to pure parallelism, without concurrency.

(1) Clone all data structures when you need to share them. Pros: very easy to understand and use. Cons: not all data structures can be meaningfully cloned, e.g., how do you clone a file descriptor?

(2) Use a static analysis (such as type checking) to enforce the precondition that writers must have exclusive access at the time of the writing. Pros: can handle both parallelism and concurrency. Cons: requires the ability to write code that a static analysis tool can understand.

Since the last one is by far the most rare approach, let me give an example of it. Quoting the Rust book:

> Here’s the rules about borrowing in Rust:

> First, any borrow must last for a scope no greater than that of the owner. Second, you may have one or the other of these two kinds of borrows, but not both at the same time:

> - one or more references (&T) to a resource,

> - exactly one mutable reference (&mut T).

https://doc.rust-lang.org/stable/book/references-and-borrowi...

Re:Why scaling and parallelism remain hard even with new too... by di4na on Wednesday August 17, 2016 @05:02AM (#12303277)
Sure but 1 and 2 end up being the same. Because when you reach the limits you clone.

The thing with parallelism is not reading. It is writing. So the best way ... is to have isolated computation then reconnect them at the end... and yes it is hard..

But parallelism has few interest in my view. Concurrency is far more interested.

Re:Why scaling and parallelism remain hard even with new too... by catnaroek on Thursday August 18, 2016 @10:47AM (#12312871)
> Sure but 1 and 2 end up being the same. Because when you reach the limits you clone.

You mean (0) and (1)? Yes, deep inside, they're the one and the same. (2) is qualitatively different from the other two.

> So the best way ... is to have isolated computation then reconnect them at the end...

What I'm saying is that Erlang-style complete process isolation is too heavy-handed and imposes unacceptable overhead in many cases. It's better to have fine-grained control over what you isolate and what you share, as in Rust.

> and yes it is hard..

It isn't easy, but rejecting large classes of errors automatically helps.

> But parallelism has few interest in my view. Concurrency is far more interested.

Besides writing correct programs in the face of shared resource usage, another of the main points to concurrency control is to safely parallelize the parts of your program that don't compete with each other for resources.