It is still not clear, it seems more like a logic error to me. Zero means success and a non zero value is an error code. It basically says "if login fails, then do this". If they would have used a boolean instead, or a result enum, the code would have taken the same path! I argue that strong typing would not have detected the logic error! The bad code was probably rushed in to make the upgrade work. All tests turning up green can really put you into a false sense of security. But that is another strongly opinioned topic!
https://objective-see.com/blog/blog_0x24.html notes that od_verify_crypt_password is called with an error check expecting 0, but on failure it returns 0x1. The code then goes on to a "success" path with an "update".
This is a classical type mismatch that C and C++ pre-2012 are so famous for. Someone writing this program where both modules were in Rust would have used Option or Result and this expectation mismatch wouldn't have existed.