GPU/FPGA programming is kind of irrelevant because as expensive as developing high performance code is for CPUs, costs go up by an order of magnitude for those platforms unless there is some existing library you can utilize (mainly applicable for ML/AI). These platforms are also very expensive. It's like saying just rent a helicopter if you need to get somewhere quickly instead of asking what "quickly" actually means and optimizing accordingly.
My point with GPU/FPGA is that if you're at the point where this level of optimization matters then you're actually dealing with situations where it's worth to invest in the big boy tools. Examples are HPC in fintech (low latency trading) and scientific computations, game development, video codecs etc. You know, "actual" computations.
Webservices are not in this category. Generally speaking with web services your goal is to "hide in the shadow of IO". If you max out your network and/or database and/or filesystem capacity, further CPU optimizations will have literally no effect. I have yet to encounter a web service where this wasn't the case.
What I do see sometimes with webservices is simply unnecessary compute, bad internal structuring, dynamic dispatch, parallelism overcommit, lack of batching, fragmented apis, fragmented data accesses etc etc all of which appear as CPU capacity saturation and also sometimes as kernelspace overhead in profiles. Async does not help solving any of these issues. And once you do solve them, you reached IO boundness and it doesn't matter anymore.
Again this is just my experience, and I'm happy to learn about what kind of web service can utilize coroutines with measurable performance benefits over a managed threadpool.
I'm not familiar with FPGA scheduling, are there any resources that explore the issue? I was under the impression it's akin to GPU compute where the main bottleneck is the bus.
1. Straw man 2. Talk about an unrelated paper 3. Conclusion: I am smart and those people are dumb
On a somewhat related note: I'd say 95% of the software I've written was IO bound(and if it wasn't at the beginning then the goal was to make it IO-bound), and that remaining 5% does not benefit at all from async/coroutines(used them in Haskell, Kotlin(Quasar) and Rust). I'm very curious about what real-world CPU-bound use cases people have that can benefit from async performance-wise. If you're optimizing on that microsecond-nanosecond scale then you shouldn't even have yields/blocks on your hot paths, and synchronization will at most be done using memfences, so what are we talking about?
I'd also conjecture that if you place your problem domain on the IO vs CPU bound axis, the problems where async would provide performance benefits can invariably be solved by other designs that perform even better (GPU/FPGA).
Yes I'm one of those people who thinks coroutines have extremely marginal actual value, and most of that value is the "feeling of how cool this is" that people experience when they first learn about the concept. If there was a way to bin the concept altogether I would do it in a heartbeat. As-is, the entire Rust ecosystem is suffering heavily because of it.