Well, one clock cycle is measured in picoseconds. That means, even if we assume 1 cycle is one nanosecond, that's 10 million CPU cycles you can do something with.

That's a lot.

Your scale is off. At the moment it is still better to think of clock cycles as nanoseconds; it's still large tenths of a nanosecond. 1 cycle per nanosecond is 1GHz, so a modern processor has 2-4 cycles per nanosecond, in which it can do a lot, but not anywhere near 10 million cycles.

My scale wasn't off, but I agree, it was quite misleading. When I said measured in picoseconds, I was thinking about 300-500 picoseconds roughly.

However, CPUs can also execute more than one instruction in a cycle so effectively, you can have more instructions than cycles.

I agree, saying that 10ms PER REQUEST is negligible is insane. If he actually read the benchmark that was referenced, he probably didn't mean what he wrote there: the benchmark measured a 10ms difference in processing an unspecified fixed number of requests from ~100 connected clients (the benchmark article isn't actually very good, and I don't care enough to dive into the github and find out what was measured).

Just totally depends. I’ve worked on systems that had to be wire-to-wire in ~5 mikes at the p99.9, and that’s crazy slow compared to the HFT assassins who are rumored to be under 100ns these days.

If you’re in single-digit mikes at the tail you’re not fucking around with someone’s green threads, and at 100ns you’re in an FPGA or even ASIC.

To serve a web page? 10ms, eh, I’d rather not spill on purpose but it’s a very, very rare professional CS:GO player who can tell. If my code is simpler and cheaper to maintain and more fun? Maybe I pay it.

What I don’t want is to ‘static bound shit to burn millis. Burning millis should buy me a beautiful sunset and a drink with an umbrella in it.

10ms is one hundred requests per second. Get ten users using your site at the same time and you are in trouble, because a single link does more than load just one request.

That's assuming no concurrency, which isn't applicable. Every public CDN will have around 10ms latency per request (because it takes time to load data from disk, fetch from upstream servers, apply WAF rules, etc). But they still handle 5 digits of requests per second.

Latency is not directly related to throughput.

> async Rust was 2x faster than threads

This phrase alone is utterly incoherent.

Not sure why you're downvoted. This is indeed nonsense. I love async Rust but I don't understand we have to revisit this topic as if there's some dire hangup or something every few months.

Author here. The benchmark part could be clearer; I acknowledge that.

Interestingly, when working with a limited number of threads, the thread approach is actually faster in that benchmark. So in practical applications, the differences are marginal and likely lean towards threads.

But even if this weren't the case, context matters. A 10ms discrepancy in a web request might be acceptable. However, in a high-performance networking application - which, let's be honest, isn't a common project for most - it could be significant.

Concurrent is rarely faster than parallel, across almost any language that supports it. If you know that you don't need obscene scalability (1000 connections is pushing the edge of what's reasonable with parallelism) then stick with parallelism. If you overuse parallelism then expect your entire system (OS and all) to grind to a halt through context switching.

You'd be surprised. 10k threads is more than manageable on Linux.

paging Dan Kegel…

Lol, fair enough, but the C10k problem these days does have a "just use OS threads" solution. It wasn't free; it took a lot of work across the industry. Computers have gotten both faster both single core and wider number of cores. And kernels have spent the last couple decades really working hard on their schedulers and kernel/user sync primitives to handle those high thread counts.

The native model falls apart under C10M, but to be fair so does traditional epoll/queue/iocp dispatching coroutines model of solving C10K. That's where you start having to keep the network stack and application data plane colocated in the same context as much as possible. That can be done with something like DPDK to keep those both in user space, or Netflix is known for their FreeBSD work making KTLS and sendfile kiss in order to keep the data plane completely in the kernel.

If you would measure pure latency between a single request (HTTP, RPC, whatever), the latency difference between any async or non async implementation should be microseconds at most and never milliseconds. If its more, then something with the implementation is off. And as you mentioned threads might even be faster, because there is no need to switch between threads (like in a multithreaded async runtime) or are there needs for additional syscalls (just read, not epoll_wait plus read).

async runtimes can just perform better at scale or reduce the amount of resources at scale. Where "at scale" means a concurrency level of >= 10k in the last benchmarks I did on this.

> 10ms... this about as long as PHP takes to start.

About that. Modern PHP in event loop with Swoole have response times of <1ms.

So 10ms is some legacy Apache/nginx mod_php/php-fpm number.

And even traditional php-fpm+Linux response times is around 2-5ms depending on configuration.