Examples of the same concept in different contexts: WTF lock Folly locks ABSEIL locks Rust mutexes
I have more interesting projects to work on than glibc maintenance.
That's one of the big problems with pthread mutexes: they try to pack too much functionality into a single type. A pthread mutex can be recursive, so every mutex needs space to store a recursion count, even though most mutexes aren't recursive. A pthread mutex can have an arbitrary priority ceiling (pthread_mutexattr_setprioceiling), so every mutex needs space to store the priority ceiling, even though most mutexes don't set that attribute. A pthread mutex can be "robust" (pthread_mutexattr_setrobust), which at least under glibc means that every mutex needs previous/next pointers in it so it can be stored in a per-thread linked list, even though most mutexes are not robust. That's an excessive level of generality, when different kinds of mutexes could have just been different types!
The other big problem with pthread mutexes is just that they are old. I could be wrong, but my impression is that much of the interest in smaller mutexes has come about relatively recently, at least compared to the age of these APIs. It might be possible to shrink pthread_mutex_t to some extent despite the aforementioned issues, but it's impossible to do so on any existing operating system (at least on existing architectures) because changing the size of pthread_mutex_t would break ABI.
Recursive locks are generally acknowledged as a terrible design pattern.
You don’t get to simultaneously claim that glibc has an optimal lock implementation and that high performance applications should implement their own. That implies the glibc implementation isn’t actually optimal because you’re paying a penalty for features that aren’t actually necessary.
Pthreads are fine as a default for most applications but then most applications shouldn’t be using threading primitives anyway and we should be using higher levels of abstraction that user better locking primitives under the hood.