Don’t roll your own crypto. Ok, that makes sense.
Don’t implement existing crypto algorithms. Ok, arguments sound good.
This leaves the options of using existing libraries. Cool. So let’s go find an existing library to deal with validating an SSL certificate chain in Python.
PyCrypto (https://www.dlitz.net/software/pycrypto/api/current/) - hmm, so basically the OpenSSL API, probably a minefield unless you are really up on crypto.
M2Crypto (https://gitlab.com/m2crypto/m2crypto) - umm, so no real docs. The recommendation is to go read a book about network security with OpenSSL. So after reading a network security book, I should be good to wire together some OpenSSL, right?
cryptography (https://cryptography.io/en/latest/) - sweet, we have docs! Hmm, so for humans we have Fernet and the ability to look at X.509 certificates. But nothing about validating. Oh, but there is a stalled PR (https://github.com/pyca/cryptography/pull/2460) from 14 months ago, to verify a certificate signatures.
Which then leaves the typical programmer to – hack something together?
So it is (practically) 2017, and using one of the most popular programming languages, you can’t verify an SSL certificate in a sane way without becoming journeyman cryptographer.
I like crypto, but using cryptography to integrate with existing protocols/standards sucks. Can we really not have an end-user focused implementation of RFC 5280 with a few knobs to turn?
Perhaps people would stop "rolling their own" crypto if there were half-decent, maintained, documented solutions out there. Maybe some day!
For transport security, HTTPS or SSH. Why go lower level?
Cert based stuff sucks, and cert management sucks. All of these suck harder because of backwards compatibility. Slowly, people are coming round to the fact you need to just deprecate stuff and get on with our lives. But the most sucky thing is OpenSSL.
Sometimes, I think OpenSSL has actually done more damage than good. You accept a shitty library because everybody uses it so it must be secure, right? Wrong. And the API is so hostile, the docs are awful. Most things that use it or try to replace it are awful (M2Crypto, PyCrypto, PyOpenSSL, even urllib3 [0]), as if the awfulness of OpenSSL seeps into your thinking - I know it's happened to me once or twice, just interfacing with OpenSSL. Even cryptography suffers from being based on OpenSSL.
Every time I have to use OpenSSL on a new project, I can't wait to see it die.
[0] https://github.com/shazow/urllib3/blob/master/urllib3/util/s...
Because not everything on this green earth uses HTTPS and SSH.
Examples:
(1) I want to implement a SAML consumer, and per the spec, I need to verify signatures. Crypto.
(2) I want to use a client-side cookie so that users can remain authenticated in the current browser session. Crypto.
(3) I want to issue a URL with a signed assertion that the owner of the content has granted permission to access it. Crypto.
Asymmetric encryption is not the be-all end-all of cryptography.
Does using certs still suck though? I get that it's a hard problem, but cert revocation and distribution, that's really hard even for OSes.
If you install the dependencies, requests uses the package cryptography to use openssl to validate certificates, by running that code.
Note that openssl does not validate certificates the way Mozilla Firefox does, even if they use the exact same root CA bundle, because some of the validation logic is code in NSS instead of data (e.g. certificate transparency requirements). Ryan Sleevi wrote on tweeter that Red Hat are working on doing something to let applications using openssl use Firefox's full cert validation logic.
This only underscores/validates wbond's critique though. Apparently the advise on how to figure this stuff out isn't even through "RTFM", docs or an easy to use library, it's "go look at the dependency of this other library which uses that other dependency that you want to use on how it is done". And then copy-paste that and hopefully not get anything wrong along the way.
That said, does the proverbial "you" know all of the chain validation features that are NOT implemented by OpenSSL when using code such as in urllib3? What if you actually care more about revocation than utmost in connection performance? How does one trust a custom CA root? What if you want to verify a cert chain for something other than a TLS connection?
My main point is: here we are with people needing crypto, and (it seems) no one has taken it upon themselves to write good crypto libraries for (some of) the types of tasks that are fairly common. The obvious exception being NaCl. However, that has issues with pragmatic things like distribution due to CPU optimizations, hence libsodium.
You might need to care a whole lot about X509 when dealing with loading your certificates and the complete chain though. But overall if you have a nice python wrapper that shouldn't be too bad. Not pleasant, but not too bad.
We do have several quite pleasant cryptography libraries like libsodium, SJCL, etc, but none of these deal with x509.
So using crypto code will be good once we rid the world of contemporary code and move into the future where all protocols are implemented on top of poly1305, ed25519 and chacha20. Considering how long it took to get rid of SSLv2 and v3, I just think perhaps some effort should go into making good, usable solutions for crypto that needs to be used now.
Thus you have proven his point.
The key the CA uses to sign OCSP must be held safely because it is important, even if that key can't sign new certificates. I think it belongs in an HSM.
I've read Let's Encrypt spends ~98% of the time of its intermediate signing keys on OCSP, not on new certs. If the OCSP was good only for an hour instead of a week, they would need to perform many more signatures per unit of time which would require more hardware.
And anyway, that's the server case. OP just wanted to check the revocation as a client as far as I can tell, which definitely doesn't require any extra hardware.
My use case involves embedding it, but also being able to verify it (Adobe distributes trusted CA certs through a signed PDF). Never would I need to run my own PKIX infrastructure.
(To be fair, it can be done without certificate chains, but it could also be done with chains.)
C/C++, Ruby, JS, etc. are much, much worse in usability.
And Python:
https://github.com/pyca/pynacl
This is one of my projects using these libraries, a PGP replacement based entirely on the NaCl API:
Simpler than Openssl or whatever, but still not ideal.
One of the main frustrations is you don't know if the output is correct until you manage to reproduce some known key/document exactly.
When you're debugging anything else, you can sort of see what's wrong as you come closer to a correct solution. A crypto scheme is only correct (you've put the right things in the right places) when it's exactly right. Until then, you have a jumble of characters.
It took me ages to match ECDSA on java with Python. In the end, it was because of some fine print where I needed to hash one version but not the other before signing.
I was doing some bitcoin calculations and needed it.
Otherwise, though, it's an inferior and outmoded signature system. It's got the worst random nonce dependency of any modern crypto primitive: if it's even biased, just a little bit, you can recover keys from groups of signatures (a full repeat instantaneously destroys security with a single pair of signatures). It's weak against simultaneous attacks on multiple signatures, so it minimizes the effort attackers need to spend. Meanwhile, it's inefficient compared to the alternatives, so it tends to maximize the effort you have to spend. It's also hard to implement without side channels.
Modern cryptography engineers would recommend something like EdDSA instead.
People have been misconstruing the bikeshedding and cliquishness of the IETF as enemy action for decades.
John Gilmore has a story about how, during IPSEC standardization, someone was pushing for a CBC chained-IV construction hard, and that he was both confident it could only be enemy action and had sources suggesting it was. This came out right after the Snowden leaks, so everyone took it seriously.
But if you look at it context, I'm pretty sure the people he was talking about were Perry Metzger and Bill Simpson† (both clearly not NSA plants). They were arguing with Phil Rogaway --- calling one of the most famous and prolific cryptographers a "so-called" cryptographer when he cautioned them not to do dumb things like chaining IVs.
There's a message thread you can look up on the Internet where this happened. Rogaway even got a petition put together from a bunch of other cryptographers, including Rivest. No luck! The IPSEC standards committee ignored them.
A decade or so later (earlier, really, but nobody took Bard's paper seriously) we discovered chained CBC IVs lead to the BEAST attack on TLS.
Enemy action? No. Crypto standards groups don't need enemy action. They are intrinsically evil, and need to be avoided.
† I think this is the case, but I haven't confirmed it with Gilmore; maybe he's talking about a different controversy during IPSEC standardization. But these are the ones where the details fit from what I can tell.
>a full repeat instantaneously destroys security with a single pair of signatures
Roughly--assuming ECDSA parameters (H,K,E,q,G)--where H is a hash function, E the Elliptic Curve over finite field K w/ point G of prime order q. Suppose two different messages m and m' have been signed with private key x using the same (non-ephemeral) random nonce value of k.
According to ECDSA Signing these messages m, m' become signatures (r,s), and (r',s') where;
r = r' = kG,
s = (H(m) + x*r)/k mod q,
s' = (H(m') + x*r)/k mod q.
Observe that, (H(m) + x*r)/s = k = (H(m') + x*r)/s' mod q.
Or, x*r(s' - s) = s*H(m') - s'*H(m) mod q.
Which allows us to recover the private key x.Since,
x = s*H(m') - s'*H(m) / r*(s' - s) mod q.That still ends up being 'rolling your own crypto' and mostly the kind of crypto the article is talking about. The author is describing putting together cryptographic constructs from the sort of low-level pieces provided by your typical runtime crypto library. This is probably the far bigger problem with the admonition - it's vague enough that someone might think that since they're not actually implementing AES themselves, they're not rolling their own crypto. Stackoverflow is full of questions just like your PHP/Coldfusion problem.
The thing that saved me finally turned out to be that Bouncy Castle actually do package some example programs in with the source. So once I cloned the git repo and dug into those, I was finally able to find an example of doing exactly what I needed. But trying to piece it together from the javadocs and the other online documentation? Hell no... I'd have been working on that until the heat death of the universe. :-(
Incidentally: Noise is great, but it's also fine to use TLS, if you're just a little careful.
See also:
http://www.daemonology.net/blog/2009-06-11-cryptographic-rig...
Blanket "don't use X" statements are much more useful with context and information about why not to use X (and when using X may be required, useful, or a better choice).
Why not be complete and list the caveat, instead of incomplete and possibly lead that one in a thousand down the wrong path?
And lots of articles about best practice leave out very rare edge cases. No advice is always always right.
I don't think that's an argument in support of leaving out edge cases, and even if you disagree in general, I think security articles should be more diligent than random best-practices articles.
> Far more than one in a thousand will use it as an excuse to do the wrong thing.
If an article says "always do X" and someone does X and the implementation is subtly wrong or insecure, that's an argument to improve the article (which is what I'm suggesting).
But if an article says "almost always do X, and be aware of this edge case" and someone gets it wrong by not following instructions, then I don't think the article is at fault.
I would personally want my security advice to cover any edge case I could think of, even if only in the footnotes.
https://sockpuppet.org/blog/2014/02/25/safely-generate-rando...
Just because the article recommends seeding over using /dev/random early at boot time doesn't mean the original gist shouldn't mention these issues so that readers are aware of them.
Just saying "use /dev/urandom all the time" without mentioning the caveats we're discussing here means someone might read your article and in a (misguided) appeal to authority implement something solely on your recommendation without doing their research properly first.
I would have thought someone writing security articles would want to avoid misleading someone into implementing something that's accidentally insecure. I hope I'm not wrong.
http://www.2uo.de/myths-about-urandom/
And another:
http://security.stackexchange.com/questions/3936/is-a-rand-f...
Which backs up my point that using /dev/urandom blindly without knowing about some of the edge cases isn't a good idea, and lists those edge cases and what to do about them (which is what I suggested you do too).
> And another: > http://security.stackexchange.com/questions/3936/is-a-rand-f....
Which also lists the edge cases I'm talking about.
Hey look if you just want to say "I'm aware of the edge cases and don't want to put them in my gist for others to see" that's fine with me, but dodging the issue by claiming there are no edge cases (and then listing 3 articles which all mention the edge cases) isn't the right reply I think.
Don't be surprised if someone suggests that a gist listing security best practices list some edge cases that go along with the blind advice too. Feel free to disagree, but at least disagree honestly.
That's the easy part. You can find a library that does exactly that, and only that. Will probably be just one file, not even a library [1].
But then you'll also need padding, MAC, signatures, key distribution, web of trust, etc.
[1] Example: https://github.com/dimview/speck_cipher
Furthermore, Speck has a variable block size between 32 and 128 bits. If you pick wrong, safe-looking things like CTR+HMAC-256 become unsafe (it's not hard to overflow a 32-bit counter).
The problem is, it's not nearly enough to just do Encrypt() and Decrypt(). Only when you see that, the benefits of a real crypto library NaCL become apparent.
Additionally, "bigger is better" is not a reasonable way to pick primitives. Defaults matter. Perf matters. From experience and stats, people pick according those two vectors a lot more often than they pick along the "bigger is better" axis. When's the last time you saw an RSA-16384 signature? AES-256 might have twice as big a number as AES-128, but how much faith do you have in that extended key expansion? You use AES-128 a lot more often than you use AES-256, and you definitely use RSA-2048 a lot more often than RSA-16384.
I don't think you can responsibly insist that this was an answer to the question. It is not a secure way to encrypt messages.
Complexity begins when you try doing something more involved than just symmetric block encryption.
I think I sort of see your point, but SPECK is a weird recommendation.
How is the IV stored for example.
Don't roll your own crypto.
Once you start adding things like CTR mode, MAC, etc. the API is no longer as simple.
Padding issues most likely.
PHP might auto detect padding, while CF might not.
C# had an option that I finally got to work but the documentation specifically said not to use it; but the other choices didn't work.
I had to encrypt something in PHP then decrypt it in ColdFusion. Despite being the same algorithm, key, etc, it didn't work. Strangely, the exact same data in the other direction (CF->PHP) worked just fine.
I'd just like the libraries I use to have a simple choice: Encrypt(data,key) and Decrypt(data,key) and be done.