News for hackers. Stuff that matters.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by ioquatix on Monday August 14, 2017 @03:25AM (#15007092)
It would be interesting to compare this with coroutines/fibers, and it would also be interesting to know how latency is affected. In my experience, thread pools introduce latency.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by nemanjaboric on Monday August 14, 2017 @04:12AM (#15007226)
Trick is that nginx uses thread pools for non-blocking reading on the "fast" devices - you can't use coroutines/fibers, because on Linux is not reliably possible to read the regular file in a non-blocking way, as far as I'm aware, so that means that `read` in a single fiber would block entire thread until the data is fetched from the disk.

In the storage engine I'm developing/maintaining, I've been using fibers, for both network and disk IO, until people started noticing that, if number of the client grows, and if they ask a lot of data (generating a lot of disk traffic), the performance drops severely. I've moved to the thread pools, dispatching my disk IO operations there (although it's much problematic for the reads, as writes would be performed by kernel anyway, `fsync` and `close` would be operations that you still want to do in the separate thread): https://github.com/sociomantic-tsunami/dlsnode/blob/master/s...

edit: used right link for the freshly opensourced repo.

Re:Thread Pools in Nginx Boost Performance 9x (2015) by srean on Tuesday August 15, 2017 @04:43AM (#15016469)
A slightly off-topic question tinged with a touch of envy that you get to work on D.

Many HN readers are apprehensive of the use of D because it is garbage collected and conjecture that it is not appropriate for low latency and/or high throughput work loads. Correct me if I am wrong, but that is exactly the kind of load the sociomantic needs to address. I have thoroughly enjoyed the relevant Dconf videos but it would be great to hear some first hand account.

BTW is the move to D2 done ?

Re:Thread Pools in Nginx Boost Performance 9x (2015) by nemanjaboric on Thursday August 17, 2017 @04:56PM (#15040911)
Oh, I'm aware this is a late reply, but it's better than nothing.

Yes, what you describe is exactly the kind of load we're addressing, and we're not getting GC to be in the way, simply by making usage of the reusable memory buffers, which are allocated in the first few requests, and then always reuse/recycled & fetched from the pool, so that we're not giving a chance for GC to kick in (as explained in the great blog post series, in D, GC mark&sweep will kick in only on allocations, completely deterministic: https://dlang.org/blog/category/gc/). Ocean provides some help here: https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src... https://github.com/sociomantic-tsunami/ocean/blob/v2.x.x/src..., etc. This makes D completely suitable for these kinds of applications, and from my experience, none of D's features or the misfeatures is making such implementations hard.

D2 move is not yet completely done. We're still writing code that's compatible in both D1 and D2. For example, DHT node project (https://github.com/sociomantic-tsunami/dhtnode) and libraries on which it's depending on such as ocean (https://github.com/sociomantic-tsunami/ocean) and swarm (https://github.com/sociomantic-tsunami/swarm) are able to run in D2 with no performance penalties, but they still don't use D2-only constructs.

Re:Thread Pools in Nginx Boost Performance 9x (2015) by srean on Friday August 18, 2017 @01:17AM (#15043772)
Appreciate your reply and welcome to HN.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by nemanjaboric on Friday August 18, 2017 @05:27AM (#15044739)
Thanks!
Re:Thread Pools in Nginx Boost Performance 9x (2015) by zbobet2012 on Tuesday August 15, 2017 @12:37AM (#15015537)
You can move the the co-routine onto another thread. The go scheduler does this: https://morsmachine.dk/go-scheduler

There isn't really a "comparison" here. Co-routines/fibers could behave exactly like nginx does for IO or not at all. It all depends on the implementation.

Re:Thread Pools in Nginx Boost Performance 9x (2015) by nemanjaboric on Thursday August 17, 2017 @04:57PM (#15040925)
Yes, sorry, I've missed the point that you can move them across the threads (was sidetracked by the usual notion of them running in the same thread).
Re:Thread Pools in Nginx Boost Performance 9x (2015) by vbezhenar on Monday August 14, 2017 @05:09AM (#15007400)
Why files can't be read without blocking with Linux? There's certainly an API for this.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by nemanjaboric on Monday August 14, 2017 @05:26AM (#15007460)
Which API are you referring to? If you're referring to `aio_` that's just glibc's user-space thread-pool emulation of the POSIX's asynchronous IO + many operations needed for the truly async IO are not supported (such as `stat` and `open`). There's `io_` syscalls family, but that's not in the ready-to-use state yet, judging by the manpages and various patches being submitted every month.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by vbezhenar on Monday August 14, 2017 @06:50AM (#15007759)
I'm referring to `fcntl(fd, F_SETFL, flags | O_NONBLOCK)`. `stat`, `open` are blocking, yes, didn't think about that. Also according to [1] file operations will always block, even in non-blocking mode, so I was wrong, sorry.

1: https://www.remlab.net/op/nonblock.shtml

Re:Thread Pools in Nginx Boost Performance 9x (2015) by signa11 on Monday August 14, 2017 @07:04AM (#15007819)
> ... Also according to [1] file operations will always block ...

not quite :) look at aio(7)

Re:Thread Pools in Nginx Boost Performance 9x (2015) by signa11 on Monday August 14, 2017 @05:24AM (#15007452)
> Why files can't be read without blocking with Linux? There's certainly an API for this.

i guess you are referring to aio(7) ? posix aio on linux is implemented in glibc, and doesn't scale in presence of multiple threads. which is probably what is alluded to...

Re:Thread Pools in Nginx Boost Performance 9x (2015) by biokoda on Monday August 14, 2017 @06:39AM (#15007709)
io_ syscalls only really work on XFS and require O_DIRECT which means kernel is not caching anything. Unsuitable for web servers.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by dunkelheit on Monday August 14, 2017 @04:35AM (#15007304)
Fibers and thread pools are not exclusive - you can schedule fibers onto threads. Nginx already uses event-based state-machine approach so context switch is even cheaper than with fibers (at the price of ugly code).

What they talk about in the article is that they offload potentially blocking operations to thread pools. That's not about networking, which is already done asynchronously, but about e.g. reading from a file not present in the page cache. Note that detecting that the needed bytes are already in the page cache and thus the read wouldn't block and should not be offloaded is tricky. As far as I can see they haven't done that yet so this new feature (edit: not so new, the post is from 2015) is not without downsides.

Re:Thread Pools in Nginx Boost Performance 9x (2015) by continuations on Monday August 14, 2017 @05:53AM (#15007547)
Why can Linux do async network IO but not async file IO? Seems like they would be similar. What's so special about file IO?
Re:Thread Pools in Nginx Boost Performance 9x (2015) by crest on Monday August 14, 2017 @06:05AM (#15007601)
When non-blocking socket I/O was implemented the disk was considered to be too fast to warrant the same kind of treatment. At the API level the different is that sockets, pipes and ttys are have an additional "ready for use" state while files are always ready, because the kernel can do something to access them even if it means blocking the thread while the kernel waits for the disks to respond.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by nemanjaboric on Monday August 14, 2017 @06:22AM (#15007642)
Yes, the trick is that disk device is considered "fast", so it's not selectable - it's always ready to read/write, which is a lie. I _feel_ (and think, I don't have any data to prove this) the main problem here is the file system layer, which may or may not need to block _after_ the operation is performed, and this complexity doesn't occur with sockets/pipes.

Having a always-ready state on regular files is a problem since Linux's non-blocking is going around readiness. On Windows (and I believe on Solaris/FreeBSD), completeness model is in place, so you schedule an operation, kernel does _everything_ and then you're resumed upon completion of IO.

Re:Thread Pools in Nginx Boost Performance 9x (2015) by dom0 on Monday August 14, 2017 @06:27AM (#15007663)
The most general layer in Linux (VFS) doesn't really support asynchronicity, so changing this would require a bunch of changes to every tree and out-of-tree FS - not viable. However, the usual suspects (ext, xfs) actually use a bunch of other APIs as well, where the FS itself is often not involved in simple stuff like a read(2). This theoretically would allow async IO on files in many cases; I believe there is ongoing work in this direction, though. For now, only raw / O_DIRECT is async.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by josteink on Monday August 14, 2017 @03:43AM (#15007152)
> In my experience, thread pools introduce latency.

The headline uses a the word "performance" which is rather ambiguous.

Performance has many aspects: Latency, through-put, resource-usage (doing more with less), etc etc.

A more specific headline wouldn't hurt.

Re:Thread Pools in Nginx Boost Performance 9x (2015) by ogrisel on Monday August 14, 2017 @04:08AM (#15007218)
In the benchmarks reported in the article, both through-put and latency were significantly improved by the new thread pool system.
Re:Thread Pools in Nginx Boost Performance 9x (2015) by Cursuviam on Monday August 14, 2017 @04:11AM (#15007224)
Being specific about the area being improved (blocking operations with the example of file reads) could be nice.