News for hackers. Stuff that matters.
Re:Asynchronous IO in Rust by nostrademons on Wednesday September 16, 2015 @05:43PM (#10230112)
Are you talking about threads-the-programming-model (vs. events, callbacks, channels, futures/promises, dependency graphs), or are you talking about threads-the-implementation-technique (vs. processes or various select()-like mechanisms)? And if you are talking about threads-the-programming-model, which synchronization mechanism: locks, monitors, channels/queues, transactional memory?

If the various threads in your application don't share data, then you don't need to worry about all the pitfalls of threading. But if they don't share data and you don't care about small (~10%) performance differences, why not use processes? Then you can use a really straightforward blocking I/O model, and you get full memory isolation & security provided by the OS.

The interesting questions happen when you a.) want to squeeze as much performance out of the machine as possible or b.) need to share data between concurrent activities. Then all of the different programming models have pros and cons, and I'm not sure you can define a "best" approach without knowing your particular problem. (Which, IMHO, validates Rust's "provide the barest primitives you can for the problem, and let libraries provide the abstractions until it becomes clear that one library is a clear winner" approach.)

FWIW, my experience in distributed systems is that threads+locks is a terrible model for writing robust systems, and that once you're operating at scale, you really want some sort of dependency-graph dataflow system where you specify what inputs are required for each bit of computation and then the system walks the graph as RPCs come back and input becomes available. This lets you attach all sorts of other information to nodes - timeouts, latency statistics, error statistics, whether or not this node is required and what defaults to substitute if it fails, tracing, logging, load-balancing, etc. It also adds a huge amount of cognitive overhead for someone who just wants to make a couple database queries. I wouldn't use this for prototyping a webapp, but it's also invaluable when you have a production system and ops people who need to be able to shut down a misbehaving service at a moment's notice while still keeping your overall product up.

Re:Asynchronous IO in Rust by jerf on Wednesday September 16, 2015 @06:11PM (#10230276)
Threads+fine-grained locks sucks.

But it wasn't necessarily the threads, so much as the fine-grained locks.

I'm talking about threads-the-programming-model, where we don't throw away the gains we made with structured programming. I suppose I ought to get off my duff and finish the blog post that explains exactly what that means, but if you know what structured programming is, that's actually enough to explain what I mean. (It's just that nowadays most people don't actually know what it is, not because they have no familiarity with it, but because they have too much, and don't realize that it was not actually the "ground state" of programming but actually a hard-fought advance, which we are all collectively on the verge of spending the next, oh, 5-10 years painfully rediscovering the value of.)

Re:Asynchronous IO in Rust by srean on Thursday September 17, 2015 @02:15AM (#10231826)
> I am talking about threads the programming model

If parallelism is off the table I would be surprised by any disagreement.I doubt any one finds control inverted style of programming more pleasurable (I.e. Callbacks) than code that looks synchronous but executed async. One thing that does require watching out for is that there are no blocking calls. Green threads, coroutines, fibres seem so much nicer as a programming abstraction. Is this view even contentious?

This does not address parallelism though. These are about exchanging control between stacks of execution, only one such stack is active in an unit. There could be multiple such units though and rub lies in should they synchronize by messages (less buggy) or shared mutable state (at times faster if serialization is expensive. Think Python).

A model that I like is different units of execution ( threads) in a common address space, each hosting coroutines or their ilk with the units sync'ing exclusively by message passing but without the need of serialization because they live in a common address space.

Re:Asynchronous IO in Rust by tailhook on Wednesday September 16, 2015 @05:50PM (#10230145)
Do you have any links on the "dependency-graph dataflow system" that you are talking about? Sounds a little bit similar to what I'm trying to do, except at higher scale.
Re:Asynchronous IO in Rust by nostrademons on Wednesday September 16, 2015 @06:39PM (#10230472)
The specific systems I'm thinking of were Google-internal, but there's a close public analogue with Guice & Dagger:

https://github.com/google/guice

https://github.com/square/dagger

Imagine injecting Futures into your code. Now imagine injecting them, but having the Provider record a lot of metadata about who you were calling, how you were calling it, how long it took, whether an error occurred, letting an SRE turn off the call, etc.

Re:Asynchronous IO in Rust by acconsta on Wednesday September 16, 2015 @06:28PM (#10230403)
I've seen an OCaml library, but I'm searching for solutions in other languages:

https://blogs.janestreet.com/introducing-incremental/

http://www.umut-acar.org/self-adjusting-computation