No error handling scheme can fix that problem. For instance, if a Rust API says I should return a Result<String, ENOSYS>, I can always write code like `if(ENOSYS) { return Ok("*") } instead of returning Err(ENOSYS).
Then my code would be compatible with MUSL.
if(ENOSYS) {
return "*";
}
after seeing that other implementations use different sentries like "*0" and that the spec says they MUST return NULL. That must have felt bad too.The only reason this library did the insane thing of returning "*" is because it was extremely easy to cause crashes or false-positives (NULL == NULL) by returning NULL+errno. The reason that was easy is because C lacks a means of making error handling mandatory (exceptions, tuple returns, tagged unions, whatever).
If you return a tagged union, users might mis-handle it, it's true (https://blog.cloudflare.com/18-november-2025-outage/). But they have to handle it, which is a requirement that C APIs are incapable of providing.
Also, in things like rust you can just do result.unwrap() if you want. In general, language constructs don't really save you from bad programmers.
In practice, that's less of a risk than:
char* x = function_that_might_return_null_and_error();
dangerous_function();
if (x == NULL) {...}
Or code for which pointer identity is relevant (e.g. accessing slices/subparts of a data structure): char* x = returns_null_or_a_slice_pointer(data_structure);
char* y = other_function_that_returns_null_or_a_slice_pointer(data_structure);
if (x == y) { puts("you're fucked now!"); } x : Result<string, err> = function...
dangerous_function();
switch(x) {
Ok() ...
Error() ...
}
There's no difference between result and null other than readability (there's a mechanical transformation between the two, where you use unwrap() in rust on an unchecked null dereference, and you read errno in C instead of looking inside the Err() result variant).To be clear, I strongly prefer Result<> because of the readability, but any dumb thing you can write with NULL + errno can be written with Result (and vice versa).
But it crashes at runtime, so such a bug can exist for years with nobody noticing.
The existence of such bugs is what motivated the musl developers to return "*" instead of NULL.
So if you're strictly adhering to the spec in this case then unfortunately NULL is not an acceptable way to signal a generic error condition. Hence all the seemingly bizarre special values. (In a strange turn of events all those experts doing things that looked weird turned out to know what they were doing. Go figure.)
But all those sentinels were added for the reason that, when your code is a public API intended to be widely used by who-knows-who, it needs to be really difficult to misuse it. NULL-plus-errno is easy to misuse. Yes, it's a dangerous mistake and your own fault if you do it. But people have been making that mistake for years and don't show any signs of stopping.
The root cause here is that C is fundamentally a rotten choice for public APIs of any kind. I griped more here: https://news.ycombinator.com/item?id=50381800
Here's how to explain it in order to blame openssh:
openssh processes passwords and doesn't want to reveal through timing whether a user has a valid password (or exists at all). So far so good.
What they do if they hit a user with no valid password (i.e. "*") is that they look for another user with a valid password, to use the seed from. A bit weird already, but OK from a perspective of avoiding implementation specifics on seeds and hash algorithms. They know at this point that the result must be to deny access even if the hashed value matches, because that has is for another user.
So they put the password received over ssh into crypt() with this other user's hashed value to get a seed. And then accept it if it matches, apparently?
Homework: what happens if the password actually matches another user? Can I log in as root if openssh randomly picks my user for the fake salt, and I type my own password correctly?
https://pubs.opengroup.org/onlinepubs/009604599/functions/cr...
>Upon successful completion, crypt() shall return a pointer to the encoded string. The first two characters of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
returning a sentinel value of NULL and setting errno is pretty standard C, and shouldn't cause an error.
OP's sshhole link specifically notes that it _should_ return NULL, but some MUSL devs decided they wanted it to return trash instead, and the trash they returned instead of NULL caused this.
All because some devs were too lazy to check the result and another dev decided to be clever and "fix" NULL errors from crypt by sending back 'unmatchable' trash banking on devs not checking NULL.