I forget the exact problem with OpenSSL, but it accepts particular invalid representations of signatures that the applicable standards say it shouldn't accept, because it doesn't treat the sign bit as a sign bit. This isn't generally a problem because those numbers should never actually be negative, but it's an issue for Bitcoin because everyone needs to agree on which signatures are valid.