News for hackers. Stuff that matters.
Re:On mutex performance and WTF::Lock by btschaegg on Monday April 03, 2017 @07:14AM (#14021932)
> Apple can and does update APIs in ways that preserve old behavior for apps linked against older SDKs, specifically so old apps continue to work.

Fair point, although that still won't rule out applications that get updates (and thus still would have to be completely re-evaluated on the basis of such a case).

Also, wouldn't e.g. Homebrew also get those problems if you compile against the new SDK? (Non-Mac user here, so maybe I've got the wrong impressions on that...)

> I believe they intentionally go with fair-by-default because it's the safe choice [...]

Safety might be a big concern, but on the other hand, pthreads is a standard - if the article's right, and POSIX doesn't mandate fairness, you might still argue that this "addon" was better put into a custom solution than the other way around.

Then again, I've always suspected that strange implementations (or a total lack thereof) of POSIX must be one of the main reasons why Boost exists...

Re:On mutex performance and WTF::Lock by eridius on Monday April 03, 2017 @03:39PM (#14026350)
> Also, wouldn't e.g. Homebrew also get those problems if you compile against the new SDK?

Yes, but nearly all Homebrew software is cross-platform unix software, so that software will already have to deal with having unfair mutexes on Linux.

> … pthreads is a standard - if the article's right, and POSIX doesn't mandate fairness …

It doesn't mandate fairness, but it doesn't mandate unfairness either. Apple platforms defaulting to fair is no less conforming than Linux defaulting to unfair.

Re:On mutex performance and WTF::Lock by btschaegg on Tuesday April 04, 2017 @08:08AM (#14031646)
> Apple platforms defaulting to fair is no less conforming than Linux defaulting to unfair.

Of course, but it makes it less performant in many situations, while you can't count on that feature if you're coding against the standard (i.e. if for cross platform development; hence OP's blogpost). Which - in turn - explains why people are complaining about the performance, since they need to guarantee the fairness themselves anyway if they really need it.

On the other hand: If you're only coding against OSX, having the fairness in an extension shouldn't be a problem anyway.

Of course, this then facilitates other problems, if the standard is not evolving fast enough (the whole OpenGL extension mess would be a good example).

Re:On mutex performance and WTF::Lock by eridius on Tuesday April 04, 2017 @08:48PM (#14038710)
I don't think Apple makes decisions about their OS with the mindset of "what will someone who's writing a cross-platform CLI tool that happens to work on OS X expect?". Making mutexes unfair by default would make it behave more like other platforms that cross-platform tools expect, but at the cost of making software written specifically for OS X not be correct-by-default.

In general, when picking defaults and choosing between "correct" or "performant", the decision is usually made in favor of "correct", because incorrectness should always be an opt-in thing. I'm actually kind of surprised Linux and Windows default to unfair locks.

Re:On mutex performance and WTF::Lock by btschaegg on Wednesday April 05, 2017 @03:55AM (#14040492)
That's the point though, if you're Writing threading code specifically for Windows, you're not writing pthreads code. So all MS has to care about is that their custom threading abstraction is safe.

POSIX is more of a common denominator than a good choice in many usecases (you'll never get far if you're dealing with complicated file handling for example[1]), but it is an essential baseline for cross-platform development. If you're changing too much about it, you're burdening cross-platform devs to do system-specific implementations anyway, so there's not much point to it.

As a flipside, you'll also get devs who see a guarantee on one system, assume the standard guarantees it, and then make statements about their library that are untrue (i.e. "WTF::Lock is fair").

[1]: https://youtu.be/uhRWMGBjlO8

Re:On mutex performance and WTF::Lock by eridius on Wednesday April 05, 2017 @04:14AM (#14040559)
Just because you're using a POSIX API doesn't mean you're writing cross-platform code.
Re:On mutex performance and WTF::Lock by btschaegg on Wednesday April 05, 2017 @08:05AM (#14041434)
No, but nothing is stopping you from using another New&Better™ Apple API if you're not concerned by cross-platform compatibility. Having tons of vendor-specific code in your cross-platform software, on the other hand, is a pain and one of the main reason why standards like POSIX exist.

Conformance with a standard doesn't just happen on a whim - if you negate the benefits that come with having a standard in the first place, any criticism you'll get as the provider of said API you brought on yourself.

Edit: Also, I have to say: I really don't like the attitude of "there are other use cases, so the one you are referring to when voicing criticisms is invalid". This has never been a constructive use of anyone's time. As I said, there are very valid reasons for why the API works as it does, but that doesn't invalidate the argument that the basic decision might not have been the best one to begin with.

Re:On mutex performance and WTF::Lock by eridius on Wednesday April 05, 2017 @03:52PM (#14045021)
What New&Better™ Apple API are you thinking of when it comes to mutexes? There's no reason for Apple to duplicate the functionality already provided by POSIX just for the sake of having a proprietary API.

> Conformance with a standard doesn't just happen on a whim

I genuinely don't understand what argument you're trying to make here. Apple is in compliance with POSIX. You seem to still be arguing from the position of "Windows and Linux behave in a certain way, therefore that way is correct", which is, quite frankly, bullshit.

Re:On mutex performance and WTF::Lock by btschaegg on Thursday April 06, 2017 @02:05PM (#14052958)
It's not a point on the implentation being incorrect. I honestly don't know if you read any of this - somehow I doubt that more and more.

The original point was: Apple uses fair mutexes by default, and that slows down any algorithm not needing that feature. Your point was: But it is still correct, and yes it is. But it's also still slow, and anyone pointing that out is as much in the right as you are when you refer to correctness.

Apple then introducing an out-of-standard solution (os_unfair_lock) for that (which is the exact duplication you talk about) is also just as suboptimal and worthy of criticism, since it torpedoes the whole reason for the existence of the standard (i.e. you still get #ifdefs or unnecessary abstractions all over the place). And - as I said - I get that no-one would ever want to change an implemented default behavior in their OS, but that doesn't make the situation any better.

Something similar happened in Microsoft's C++11-implementation of std::regex: The library uses strongly recursive function calls and might even cause stack overflows on various (not all that complicated) patterns in some scenarios. That doesn't make the implementation not correct (every regex implementation has its limits, and it never produces invalid results), but it makes it a hell of a lot less usable for non-trivial problems.

So, you can harp on on the correctness thing as long as you want, but you might consider not ignoring valid arguments before throwing words like BS around. "This implementation might be better" is not BS. Aggressively misinterpreting statements can be, though.

Re:On mutex performance and WTF::Lock by eridius on Thursday April 06, 2017 @02:52PM (#14053433)
Accusing me of not reading your comment is kind of offensive, and the implication (that I'm only disagreeing because I'm not reading your arguments) is also offensive.

Given that POSIX does not define a way to select between fair and unfair locks, having any way to do that is by definition not a standard. Your argument against Apple having a way to get unfair locks is just as much an argument against Linux/Windows having a way of getting fair locks (assuming they even have that! I sure hope they do). And in any case, you don't need to #ifdef all your code everywhere to use os_unfair_lock, you just need to #ifdef a single line where you call pthread_mutexattr_setpolicy_np().

> I get that no-one would ever want to change an implemented default behavior in their OS, but that doesn't make the situation any better.

I recognize that having different behaviors on different OSes is suboptimal. The problem is you continue to imply that macOS's behavior is wrong, with the only justification being that Linux and Windows both behave differently. And I do not accept that argument. I'm also not convinced that this is really a big problem anyway. Fairness in locks is probably a little more important on Apple platforms than other platforms because of the QoS system. If you have an unfair lock that both the main thread and a "utility" thread use all the time, the utility thread will probably get starved since the main thread has a much higher QoS. So it's good that Apple platforms default to fair locks, because thread starvation is pretty bad. Personally, I think Linux and Windows platforms should default to fair locks too, with unfair-but-fast being opt-in, but that's an argument I've already made.

> but you might consider not ignoring valid arguments

Once again you are being offensive.

> Aggressively misinterpreting statements can be, though.

And even more offensive.

Seriously, if this is how you're going to argue, I don't see much point in continuing. You're providing fairly weak arguments in favor of all platforms should be the same, without recognizing that the platforms may have reasons for behaving differently, and constantly trying to imply that Apple platforms should be the ones to change while providing literally no reason whatsoever why that change should happen instead of Linux/Windows changing. This is not a compelling argument. And you keep insulting me at the same time, and I genuinely don't understand why you think that's going to change my mind.

Re:On mutex performance and WTF::Lock by btschaegg on Thursday April 06, 2017 @03:19PM (#14053690)
> And even more offensive.

Seems like we get the same vibes from each other; I have to point out though that I'm actually trying to be civil here. I honestly can't see where you are getting various of your implications from - I can't agree with half of what you state I'm implying here. Intended offensive statements would look different than that one.

So be it - we most likely both have more productive things to do. Maybe we'll have a more constructive discussion on something else someday.