Don't get me wrong: it's still easier. But there's an interesting property I've noticed of statements of the form "by definition, problem X is impossible when you have Y", which is that X-analogous problems absolutely can happen, and they are often more complicated to solve. For another example: "by definition, memory leaks are impossible when you have a garbage collector".
No, let's not. They mean different things. The execution of Python generators is concurrent with your code, but not parallel or asynchronous. And it's also completely possible to have concurrent synchronous parallel execution [1].
[1] https://en.wikipedia.org/wiki/Bulk_synchronous_parallel#The_...
And I avoid using concurrent if I can. It does not have a strict CS definition, and it is prone to miscommunication. Many people use “concurrent” when they are talking about parallelism. Some people’s CS definition of concurrency is not very well matched with the common non-CS definition of the word.
The dictionary definition of “concurrent”, according to Google, is: “existing, happening, or done at the same time.”
synonyms: simultaneous, coincident, contemporaneous, parallel
Using “concurrent” to mean not parallel, or even out of order or not necessarily parallel, just seems like a bad idea.
I'm not sure we do. The comment you replies to does not conflate those. "You can have concurrency as in async Javascript" is exactly correct: an example of concurrency is async in Javascript. People don't say "an example of asynchronicity" because that's not what they want to talk about. You can't really change the word unless you change the topic they are trying to talk about.
> And I avoid using concurrent if I can. It does not have a strict CS definition
It very much does. It means an operation overlapping in time with another one, i.e. one beginning before the other one ends. The colloquial definition is irrelevant in a technical context. It's like suggesting scientists stop using the word "theory" on the grounds that laymen don't use it properly. No, people should just use the right word in the right context.
You lose some potential speed this way, but you are able to reason about the states your program will take far more easily.
I ask because at work, we have a lot of browser tests that we'd like to start transitioning off of that deprecated queuing mechanism. If I can rip out Selenium's version and provide my own mechanism that doesn't deadlock on async/await, it would make the transition a lot easier.
https://bl.ocks.org/johnsonjo4531/99256568deaf8c0f1685793a4a...
Basically it is used like this:
```
var lock = new AsyncLock();
(async () => {
for await(var _ of lock) {
console.log("here 1a");
await sleep(1000);
console.log("here 1b")
}
})();
(async () => {
for await(var _ of lock) {
console.log("here 2a");
await sleep(1000);
console.log("here 2b")
}
})();
/*
When Lock is working should print:
here 1a
here 1b
here 2a
here 2b
Without the lock it would print:
here 1a
here 2a
here 1b
here 2b
*/
```If you want to check out the implementation go to the blocks link. (which is basically just a gist that you can run). Here's the actual gist link: https://gist.github.com/johnsonjo4531/99256568deaf8c0f168579....
It requires Chrome 63 (at this point Chrome Canary) because it uses async iterators. I used this because I can close the lock automatically in case an error is thrown, the loop is broken. It only iterates one value and only after the lock is unlocked.
The read/write combinations to the semaphore variable need not be atomic since we know that if this thread is running there are no other threads that can be reading and then modifying it.
MyObject.queueSemaphore = N;
MyObject.emitter = new EventEmitter();
MyObject.queue = function( ... ){
// if this run can take place go
// otherwise wait for a run-complete to try again
if(this.queueSemaphore > 0) {
this.queueSemaphore--;
} else {
// must return a promise that resolves and possibly requeues
return new promise((resolve, reject) => {
this.emitter.once("run-complete", () => {
resolve(true);
});
}).then(() => this.queue( ... ));
}
this.promiseBasedProcess()
// ... work
.finally(() => {
this.queueSemaphore++;
this.emitter.emit("run-complete");
});
}So users of the lock would call acquire and get back a deferred (a promise in other language), but say in the most basic way, a callback is passed to it. When lock can be acquired, that callback is called (or deferred is fired).
When done with the resource, must call the release. Release then will trigger the lock to inspect the list of available waiting clients, pick one and fire their callback letting them know they can access the resource, and so on.
https://github.com/twisted/twisted/blob/trunk/src/twisted/in...
let lock;
const log = (msg) => {
lock = new Promise(async (resolve) => {
await lock;
console.log(msg);
setTimeout(resolve, 1000);
});
};
'hello world'.split('').forEach(log);I needed it in order to use a library that had some annoying global state in it.
That you don't have the concerns of shared memory concurrency, that's actually not such a big relief, especially given the right abstractions. And the underlying problem, that you need to deal with concurrency, doesn't go away.
I've just written an article describing how JavaScript's Promise leaks memory in "then" chains and that it shouldn't: https://alexn.org/blog/2017/10/11/javascript-promise-leaks-m...
The reception was actually negative, having lots of people telling me that I'm concerned about a niche use-case, but functional programming isn't a niche at all, being one of the two known ways to write reasonable, composable code that deals with concurrency and asynchrony, the other way being to have a draconian borrow checker, like that of Rust.
Mainstream developers are in general deluding themselves that the tools they are using are adequate, in spite of repeated evidence to the contrary, whereas most current breakthroughs in productivity were born out of functional programming, starting from garbage collection, but also actual libraries that people are now fond of, like React.
Haskell does shared memory concurrency. The JVM / Scala does that too. In Haskell you work with things like IO, MVar, STM (transactional memory) and many more. High level abstractions that have been borrowed by Scala as well, my current languages of choice.
I feel much more confident writing concurrent Haskell or Scala code, with 0% test coverage, then I am writing JavaScript code with 100% code coverage.
It is thread safe from data being accessed by two different threads, well because there is only one thread running. But surprisingly it is not free from application level data races in general, and you might still need async mutexes or lock type constructs.
That is because it is possible to have multiple callback chains each partially modifying the same data structure, or trying to write to the same tmp file, or modify some global resource.
In general with callback programming each callback chain specifies a sort of a poor man's IO concurrent context of execution. In a thread you spawn a thread to execute some code in a function, so it executes instructions A, B, then C. And there can be another thread executing the same A, B, C instructions in parallel. With callbacks it's the same idea, a chain of A, B, C callbacks can execute IO requests in parallel. Two client request coming almost at the same time, would run through A, then B, then C functions. A, B, might be called by first request. Before it finishes, and calls C, another request might start A, B chain again.
Thread-based concurrency need not be overly scary. Any concurrency model will take some time to wrap your head around, but with the good ones, once you get comfortable with them, they should feel very safe, intuitive and deterministic. Scala's approach of "Futures + immutability when you don't need mutable state, or Actors when you do need mutable state" has worked well for every problem I've encountered in the last few years, and I've certainly heard great thing's about Go's approach too.
ES6 has improved the state of concurrency in JS, it's now similarly easy to read and reason about as Scala's model, but the lack of actual parallelism is a pretty big limitation - often a deal breaker for server-side code, and still often an issue client-side.
Well, in my experience, it has been a deal breaker in a select few, rare cases. But often?
Do you routinely come across problems that could NOT be solved using a multiprocess approach (in Node) or webworkers on the client side? Could you share some of these scenarios?
Even node uses a thread pool under the covers for that.
With that approach we can go back to the happy openmpi/mpich days and take the gargantuan performance hit that ended MPI clustering for any serious real time scientific processing.
Again, I must press for examples/real scenarios.
Not giving you more 'hints'. You webdevs are kind of dull anyway. I/O this and I/O that and you'll accept untrusted input until the sec industry all drive ferraris on your mistakes.
Most peoples problems come with being over clever with their per thread data/stack, priorities, affinities, attributes, etc..a simple threaded program with a couple of locks for shared data and a flat schedule is efficient and easy to understand on multi-core systems.
Event loops and aio are nice but they are not a superior standalone solution as evidenced by nodes use of libuv.
And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?
Yes.
> But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
Is not about how much work it can handle, it's about how much it can offload. Async servers can handle a much greater volume of I/O bounded tasks. So it can handle more connections. When the task is CPU bounded you can either create a thread (which does not really scale well) or offload it to some other servers that can scale horizontally (i.e doing micro services)
> And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently? Much like, say, Python Django?
Yes, to take advantage of all CPU cores you have to create more server instances. But why would they talk to the same database instance? It could be a replica or a shard. You can even have a pool of shards connections per server instance.
Sometimes you want to use a thread to keep the CPU from blocking (e.g. long database operations).
Also, sometimes you want to use the CPU to its full capabilities.
On the contrary. That's the prototypical use case for non-blocking event based IO. No threads needed.
>Also, sometimes you want to use the CPU to its full capabilities.
You can use all CPUs by using multiple processes. That's not an issue.
Threads are useful when you want to run multiple algorithms in parallel on the same bigish in-memory data structure, especially one that has a lot of pointers.
Something like an in-memory graph database or a desktop application that lets you work with huge files in-memory, or even complex user interfaces. For instance, I'm not convinced that the cross process bridging that React Native has to do is a great idea.
So yes there are use cases where threads are very beneficial. But on the server side it's essentially the database/analytics systems themselves, not the code that accesses them.
> The problem with offloading to a thread is that you can't structurally share data with another thread.
That's really a JS issue, not a general async programming issue, though.
> Yes.
Just to expand, this (like most things) is a simplification, as I'm sure nitely is just being too brief to explain. It's true for Hello World, and a little further, but real-world web servers in non-trivial contexts typically utilise techniques like clustering, workers, and other ways of delegating tasks to external processes.
Why not? Unless you've exceeded the capacity of a single DB and have a real use for sharding etc, it would not make sense to have difference DBs (+ replication overhead) for different Node processes.
You can have a 12-processes node cluster and still not need a second db.
Yes -- node is by default a single threaded, single process server.
>But doesn't running on a single thread put an upper limit on the amount of work a server can handle?
Not any more than this is the case with Python, PHP, Rails, etc -- which also don't do multi-threaded (or don't do it well and not by default), and which on top of that don't have asynchronous capabilities (again, not by default) and are even less performant than a single Node app.
Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).
>And, if the solution is to spin up more servers with access to the same database, doesn't that mean that we are now having multiple threads accessing the database concurrently?
Databases take care of serialization of multiple queries for you -- and for more complex cases (with or without transactions for fuller control).
uWSGI makes running a threaded or multi-proc python webapp trivially easy (and as of Python v3.6 async comes as standard)...
>Which is why a simple Node running with its single process and single threaded execution can e.g. beat a Python server with two dozens of workers (e.g. gunicorn) in handling simultaneous connections (assuming Node code is properly async in the most part).
Node can beat a threaded python app for sheer volume of concurrent connections to clients, yes. But for a lot of traditional backend work (e.g. talking to a DB) async is no faster (indeed it's often slower) than a threaded approach.
Node (or async in general) is great for terminating inbound client connections; talking to local, in-memory caches or making backend calls to remote, non-local REST services.
For making local DB connections or doing any CPU work (e.g. parsing XML docs returned from an API) single-threaded async rarely yields better performance over threads/multi-proc. A good illustration of this is pgbouncer (async on the client facing end; threaded - i think? - on the db facing side).
Basically, all node really does is reduce the number of front end app servers you need to serve X incoming client connections. Just because node can handle a high concurrent connection count doesn't mean the rest of your backend services can. Regardless of whether connections originate from a single node instance or a large fleet of php/python/rails servers; you still need reverse proxies like pgbouncer/haproxy/twemproxy/squid to manage and shape those connections before they get to things like your DB or internal micro-service APIs.
Because node is single-threaded you also need to keep a very close eye on any CPU bound activity to avoid blocking all your connections. This is not always obvious and can crop up in unexpected ways (see: https://news.ycombinator.com/item?id=15477419)
Why isn't PHP etc called single-threaded then? I thought the difference was in fact that PHP fires up new threads for each connection.
PHP doesn't get to decide what happens for each new connection. That's determined upstream, and there's a lot of different pieces of software people choose to do that. Could be directly a webserver like apache's mod_php, or something like fastcgi, php-fpm, etc.
All of those front-ends chose to implement a model of a php process pool. Multiple processes, each with a single php interpreter running. Incoming connections are sent to a process in the pool. So, if the pool is 5 processes, and you get a 6th concurrent connection, that one waits.
However, you can write a C program with a single process and distinct php interpreter per thread. One example: https://github.com/basvanbeek/embed2-sapi
To date, nobody has taken something like that and done the work to connect it to a front end handler.
It acts as an intermediary between your application code and the kernel, notifying you as soon as some IO action was completed by the kernel via an event callback.
This notification is provided to you as a single queue of events; the event loop is hence single threaded.
The important thing this facilitates is making it easier to reason about your application code, since you can be assured that only one of hundreds of callbacks in your application can be running at any one time.
Does it put an upper limit on the amount of work a single server process can handle? Depending on your use case, possibly yes. NodeJS shines when most of the work each call to your server involves mostly IO, i.e. are IO bound tasks.
If on the other hand, if any of the calls are CPU bound (some complex mathematical calculations say), you're probably going to hit this limit much sooner.
Even in cases where you have to run CPU bound tasks, it is far 'simpler' to offload these to an entirely different process that uses some sort of IPC to run the calculations and communicate the completed results back to your main server process, rather than spinning up a new thread in your server process to handle those CPU-bound tasks.
Does it now not possibly involve multiple threads accessing the database concurrently? Well, yes. Databases though are rather good at handling races. Most mainstream databases provide some sort of locking mechanism to make sure that some shared record cannot be erroneously modified by two processes at the same time.
If database locks are not for you, there are other solutions possible for these kinds of issues as well. By implementing a proper message queue, you can filter out calls that access this shared record into a separate synchronous queue, while all the other calls can be made to the DB simultaneously.
Why bother with mutex locks and races in your application code when other people (authors of libev/databases) are willing to do it for you?
Yes, which leads to hilarious things like: https://medium.com/walmartlabs/using-electrode-to-improve-re...
"In our tests, however, we found that React’s renderToString() takes quite a while to execute — and since renderToString() is synchronous, the server is blocked while it runs. Every server side render executes renderToString() to build the HTML that the application server will be sending to the browser."
"The average renderToString()call with this configuration took 153.80 ms."
JavaScript is single threaded. Node.js is not.
Node executes the entire JavaScript code in a single thread. However, the I/O requests dispatched by the JavaScript code (file I/O, network I/O, db I/O) can be executed in separate threads.
That is why Node is efficient for applications with lot of I/O (typical web apps), and can still provide a default thread safety for global objects.
But I think, it will not work efficiently for number-crunching CPU intensive applications.
https://medium.com/the-node-js-collection/what-you-should-kn...
Having a single threaded (asynchronous) server simply displaces the abstraction of threads.
Please feel free to correct if I'm wrong about this...
Whether a thread is "virtual" - I don't know what that means to you, but threads are a primitive the OS provides. Some CPUs have hardware support for threads, and some don't. So, I suspect they are less virtual than you're thinking.
A multicore cpu is wasted on single threaded program unless you are lucky enough that your problem is so embarassingly parallel that can be handled by N indipendent the processes
It's one less complication in your logic when you don't have to think "what if some other thread modifies this data while I am doing it?". And you never get hard-to-detect and hard-to-fix errors caused by that nor do you need to try to set up locks that might cause performance problems.
So some might say that threads and locks and synchronization primitives are the solution while others might say they are the problem.