News for hackers. Stuff that matters.
Re:Measuring mutexes, spinlocks and how bad the Linux schedu... by tsimionescu on Wednesday January 01, 2020 @05:38AM (#21927513)
Approaches using IPC/sockets are significantly more complicated than shared-memory parallelism in most programming languages. Sockets at least are also an absurdly wasteful communication mechanism in low-concurrency situations, compared to direct shared memory access.

If you have a simple REST web server, single-thread is simply not a realistic option. The next simplest thing is to do shared memory parallelism, which works perfectly for a few dozens to low hundreds of threads. That already covers a lot of scale for many realistic problems. Moving from that to a distributed system design is a significant jump in implementation complexity, and will likely increase per-request latency with most straight-forward implementations.

Re:Measuring mutexes, spinlocks and how bad the Linux schedu... by cryptica on Wednesday January 01, 2020 @06:39PM (#21931618)
>> which works perfectly for a few dozens to low hundreds of threads

It's a lot more complicated that than. It depends heavily on how many threads are writing to the shared memory and how often. If the workload is write-heavy from many threads then then it can really add up. Also, if there are multiple shared memory locations which all use mutexes, it adds up as well. Shared memory offers diminishing returns as the logic becomes more complex.

Often, it's better to simply design the system in such a way that it removes or at least minimizes interactions and data sharing between different threads or processes. With such designs, often the overheads of IPC communication become very low.

Some engines like Node.js are focused around IPC as the main mechanism for sharing data between different processes. It produces relatively elegant code which is easy to follow and this design makes it relatively easy to identify the bottlenecks.

I agree that the approach I was suggesting is probably not ideal for low-concurrency situations (like games as I mentioned in my last comment) but I still think that shared memory access mechanisms are not elegant. That said, I find that programming solutions which try to squeeze every last drop of performance out of the hardware are rarely elegant architecturally.

Re:Measuring mutexes, spinlocks and how bad the Linux schedu... by imtringued on Wednesday January 01, 2020 @07:47AM (#21927821)
In my experience it is very to easy get into situations where using only mutexes is impossible. As soon as you need a lock per object and access two objects then you have to make sure to only acquire one lock at a time. The end result is that you are effectively using the stack as a message queue that can only hold one element.
Re:Measuring mutexes, spinlocks and how bad the Linux schedu... by tsimionescu on Thursday January 02, 2020 @05:55AM (#21934390)
The solution with multiple locks is to always take them in the same defined order on any code path - that guarantees that you can't get into deadlock situations.