You have to be careful with those things - for instance, you have to special case whether you'r on a single CPU or multiple CPU system, because a spin lock will block forever in a non-preemptive context, such as the kernel.

Outside of hard realtime code, there's zero reason to use spin locks.

In hard realtime code, a FIFO mutex gives you bounded wait time because you have a fixed number of threads.

Interesting read: http://www2.rdrop.com/~paulmck/realtime/SMPembedded.2006.10....

> Outside of hard realtime code, there's zero reason to use spin locks.

That is just not true. You _must_ use them in the case where the kernel is non-preemptable. Additionally, if the locked resource is held for a very short time, a spin lock is likely a more efficient choice than a traditional mutex.

> Outside of hard realtime code, there's zero reason to use spin locks.

On some common architectures, releasing a spin lock is cheaper than releasing a mutex.

On all architectures, releasing a mutex requires at least a branch (to see if you need to wake up sleeping threads) that you don’t need with a pure spinlock.

But if you don’t have a guarantee the lock owner won’t be preempted, well, spinning for a whole timeslot is quite a bit more expensive…

Spin locks are a tough sell in a preemptable context. Say you have two processes that share some memory location. They both briefly access it for a very short time, so you protect it with a spin lock. Well, what happens in the case when one of the threads is preempted while holding the lock? The other thread would try to aquire it, and just spin for its entire timeslot. No bueno. When you call spin_lock() in the kernel it actually disables preemption until you call spin_unlock() to avoid this. You can't disable preemption from userspace. There might be a use for a spin lock if you have a process that is SCHED_RR, but I haven't seen it.

You are not wrong in general but note that there are ways to disable preemption in userspace in practice, be it SCHED_FIFO[1] or isolcpu with cpu pinning.

[1] you better be careful with spinlocks and priorities here as you can livelock forever.

Can't upvote this enough. If you are using spinlocks because "mutexes are slow", please reconsider since the contended case makes much more sense with mutexes than spinlock, and the uncontended case is exactly the same.

And a mutex that you never take is cheaper than one that you do. Taking the (f)mutex is fast but any context switch will be brutal if you're really using threading to increase your performance.

That said I'm used to situations where we're pinning thread affinity to specific cores and really trying to squeeze out what you can from fixed resources.

Pinning threads to cores with affinity is another technique that should be used with caution. Under normal circumstances the kernel will run the same thread on the same core as much as possible because CPU migration is expensive.

Setting CPU affinity will ensure that you always get the same core, but it might not increase performance and could adversely affect other parts of the system.

CPU affinity is a good fit for continuously running things like audio processing or game physics or similar. It's not good when threads are blocked or react to external events.

In most cases it's just unnecessary because the kernel is pretty good in keeping threads on cores anyway.