News for hackers. Stuff that matters.
Re:Rolling Your Own Crypto by wbond on Tuesday December 20, 2016 @02:27PM (#13223063)
I found the issue to be more insidious.

Don’t roll your own crypto. Ok, that makes sense.

Don’t implement existing crypto algorithms. Ok, arguments sound good.

This leaves the options of using existing libraries. Cool. So let’s go find an existing library to deal with validating an SSL certificate chain in Python.

PyCrypto (https://www.dlitz.net/software/pycrypto/api/current/) - hmm, so basically the OpenSSL API, probably a minefield unless you are really up on crypto.

M2Crypto (https://gitlab.com/m2crypto/m2crypto) - umm, so no real docs. The recommendation is to go read a book about network security with OpenSSL. So after reading a network security book, I should be good to wire together some OpenSSL, right?

cryptography (https://cryptography.io/en/latest/) - sweet, we have docs! Hmm, so for humans we have Fernet and the ability to look at X.509 certificates. But nothing about validating. Oh, but there is a stalled PR (https://github.com/pyca/cryptography/pull/2460) from 14 months ago, to verify a certificate signatures.

Which then leaves the typical programmer to – hack something together?

So it is (practically) 2017, and using one of the most popular programming languages, you can’t verify an SSL certificate in a sane way without becoming journeyman cryptographer.

I like crypto, but using cryptography to integrate with existing protocols/standards sucks. Can we really not have an end-user focused implementation of RFC 5280 with a few knobs to turn?

Perhaps people would stop "rolling their own" crypto if there were half-decent, maintained, documented solutions out there. Maybe some day!

Re:Rolling Your Own Crypto by guitarbill on Tuesday December 20, 2016 @05:24PM (#13224783)
> Perhaps people would stop "rolling their own" crypto if there were half-decent, maintained, documented solutions out there. Maybe some day!

For transport security, HTTPS or SSH. Why go lower level?

Cert based stuff sucks, and cert management sucks. All of these suck harder because of backwards compatibility. Slowly, people are coming round to the fact you need to just deprecate stuff and get on with our lives. But the most sucky thing is OpenSSL.

Sometimes, I think OpenSSL has actually done more damage than good. You accept a shitty library because everybody uses it so it must be secure, right? Wrong. And the API is so hostile, the docs are awful. Most things that use it or try to replace it are awful (M2Crypto, PyCrypto, PyOpenSSL, even urllib3 [0]), as if the awfulness of OpenSSL seeps into your thinking - I know it's happened to me once or twice, just interfacing with OpenSSL. Even cryptography suffers from being based on OpenSSL.

Every time I have to use OpenSSL on a new project, I can't wait to see it die.

[0] https://github.com/shazow/urllib3/blob/master/urllib3/util/s...

Re:Rolling Your Own Crypto by paulddraper on Tuesday December 20, 2016 @05:39PM (#13224930)
> For transport security, HTTPS or SSH. Why go lower level?

Because not everything on this green earth uses HTTPS and SSH.

Examples:

(1) I want to implement a SAML consumer, and per the spec, I need to verify signatures. Crypto.

(2) I want to use a client-side cookie so that users can remain authenticated in the current browser session. Crypto.

(3) I want to issue a URL with a signed assertion that the owner of the content has granted permission to access it. Crypto.

Asymmetric encryption is not the be-all end-all of cryptography.

Re:Rolling Your Own Crypto by wbond on Tuesday December 20, 2016 @05:40PM (#13224941)
I'm not going lower level – in fact, I'm not even dealing with a communication protocol. Certificates are also used for document and executable signing.
Re:Rolling Your Own Crypto by guitarbill on Wednesday December 21, 2016 @01:35AM (#13227229)
Yeah, fair enough - it was more of a general assertion that it's easy to forget that high-level solutions exist. It's NIH.

Does using certs still suck though? I get that it's a hard problem, but cert revocation and distribution, that's really hard even for OSes.

Re:Rolling Your Own Crypto by wolf550e on Tuesday December 20, 2016 @02:35PM (#13223170)
Look at this code from urllib3 (requests bundles this code): https://github.com/shazow/urllib3/blob/master/urllib3/contri...

If you install the dependencies, requests uses the package cryptography to use openssl to validate certificates, by running that code.

Note that openssl does not validate certificates the way Mozilla Firefox does, even if they use the exact same root CA bundle, because some of the validation logic is code in NSS instead of data (e.g. certificate transparency requirements). Ryan Sleevi wrote on tweeter that Red Hat are working on doing something to let applications using openssl use Firefox's full cert validation logic.

Re:Rolling Your Own Crypto by daenney on Tuesday December 20, 2016 @03:58PM (#13223894)
> Look at this code from urllib3 (requests bundles this code): https://github.com/shazow/urllib3/blob/master/urllib3/contri....

This only underscores/validates wbond's critique though. Apparently the advise on how to figure this stuff out isn't even through "RTFM", docs or an easy to use library, it's "go look at the dependency of this other library which uses that other dependency that you want to use on how it is done". And then copy-paste that and hopefully not get anything wrong along the way.

Re:Rolling Your Own Crypto by wbond on Tuesday December 20, 2016 @03:08PM (#13223473)
(Just to note, this wasn't me looking for help in how to accomplish that task.)

That said, does the proverbial "you" know all of the chain validation features that are NOT implemented by OpenSSL when using code such as in urllib3? What if you actually care more about revocation than utmost in connection performance? How does one trust a custom CA root? What if you want to verify a cert chain for something other than a TLS connection?

My main point is: here we are with people needing crypto, and (it seems) no one has taken it upon themselves to write good crypto libraries for (some of) the types of tasks that are fairly common. The obvious exception being NaCl. However, that has issues with pragmatic things like distribution due to CPU optimizations, hence libsodium.

Re:Rolling Your Own Crypto by problems on Tuesday December 20, 2016 @04:10PM (#13224028)
Well, this is a very particular case and it's only so bad because you're dealing with SSL and X509. X509 is a clusterfuck of its own. OpenSSL or OpenSSL-esque (like LibreSSL or BoringSSL), but in OpenSSL this essentially just boils down to X509_verify_cert, you really don't need to think much about the crypto.

You might need to care a whole lot about X509 when dealing with loading your certificates and the complete chain though. But overall if you have a nice python wrapper that shouldn't be too bad. Not pleasant, but not too bad.

We do have several quite pleasant cryptography libraries like libsodium, SJCL, etc, but none of these deal with x509.

Re:Rolling Your Own Crypto by wbond on Tuesday December 20, 2016 @04:27PM (#13224195)
Or PKCS#12, or PKCS#8, or ECDSA (OpenSSL has determinism now, but different than RFC 6979), or ASN.1. Basically anything where you are using crypto as part of another protocol, not simply for encrypt/decrypt, sign/verify.

So using crypto code will be good once we rid the world of contemporary code and move into the future where all protocols are implemented on top of poly1305, ed25519 and chacha20. Considering how long it took to get rid of SSLv2 and v3, I just think perhaps some effort should go into making good, usable solutions for crypto that needs to be used now.

Re:Rolling Your Own Crypto by TwoBit on Tuesday December 20, 2016 @10:54PM (#13226670)
"but none of these deal with x509"

Thus you have proven his point.

Re:Rolling Your Own Crypto by problems on Wednesday December 21, 2016 @05:12PM (#13233113)
His point was that crypto wasn't pleasant. In reality it's that x509 isn't pleasant.
Re:Rolling Your Own Crypto by pvg on Tuesday December 20, 2016 @03:14PM (#13223510)
Serious question: what are the common, practical situations you've run into or can think of where you need to be validating certificate chains yourself in your application code?
Re:Rolling Your Own Crypto by wbond on Tuesday December 20, 2016 @03:15PM (#13223520)
Document signing is what pushed me down that path. It uses the same ecosystem and technologies as TLS, but cares more about revocation.
Re:Rolling Your Own Crypto by pvg on Tuesday December 20, 2016 @03:22PM (#13223572)
Ah I see. Seems a bit off the beaten path already, though. Were they some sort of existing documents, generated by something not easily available server side?
Re:Rolling Your Own Crypto by wolf550e on Tuesday December 20, 2016 @03:58PM (#13223890)
Did you require online (not cached) revocation check? That would require buying more HSMs.
Re:Rolling Your Own Crypto by wbond on Tuesday December 20, 2016 @04:36PM (#13224278)
No, I'm just implementing code that signs and verifies PDF digital signatures for compliance with EU regulations. I'm not dealing with HSMs, or even the PKIX infrastructure other than verifying a certificate is valid according to Adobe/ETSI and generating/checking signatures. The certificate issuers need to provide the CRLs and OCSP responders/responses.
Re:Rolling Your Own Crypto by wolf550e on Tuesday December 20, 2016 @08:51PM (#13226184)
Why is this downvoted? Could the downvoter please comment?
Re:Rolling Your Own Crypto by viraptor on Wednesday December 21, 2016 @01:50AM (#13227293)
Online revocation check doesn't require buying HSMs. HSMs also usually operate on a much lower level / aren't really connected to the internet.
Re:Rolling Your Own Crypto by wolf550e on Wednesday December 21, 2016 @06:11AM (#13228262)
If you can't use a cached OCSP response and instead need a fresh one (or the cache is valid for a short period only), that means the CA must sign a new OCSP response much more often.

The key the CA uses to sign OCSP must be held safely because it is important, even if that key can't sign new certificates. I think it belongs in an HSM.

I've read Let's Encrypt spends ~98% of the time of its intermediate signing keys on OCSP, not on new certs. If the OCSP was good only for an hour instead of a week, they would need to perform many more signatures per unit of time which would require more hardware.

Re:Rolling Your Own Crypto by viraptor on Wednesday December 21, 2016 @06:23AM (#13228307)
But that's an implementation detail. Sure, some HSMs will make signing faster, some won't. You don't need to use HSMs which will have a considerable markup too. You can use more servers if you prefer.

And anyway, that's the server case. OP just wanted to check the revocation as a client as far as I can tell, which definitely doesn't require any extra hardware.

Re:Rolling Your Own Crypto by wbond on Wednesday December 21, 2016 @08:26AM (#13228866)
In some cases revocation info will be live, in others it will be stored inside of the document.

My use case involves embedding it, but also being able to verify it (Adobe distributes trusted CA certs through a signed PDF). Never would I need to run my own PKIX infrastructure.

Re:Rolling Your Own Crypto by paulddraper on Tuesday December 20, 2016 @05:41PM (#13224944)
I want to implement SAML (as a consumer).

(To be fair, it can be done without certificate chains, but it could also be done with chains.)

Re:Rolling Your Own Crypto by pvg on Tuesday December 20, 2016 @09:44PM (#13226423)
Yep, I saw your more general list further upthread as well. I'd argue that much of the time these things are bad and essentially a kind of unnecessary, self-inflicted cryptography. In most cases the same things can be accomplished with a secure authenticated channel, someone hanging on to the relevant state and a cryptographically secure random id. A prime example is all the 'state-carrying authenticated/encrypted session cookie' infrastructure typically built into ruby and python web frameworks.
Re:Rolling Your Own Crypto by paulddraper on Tuesday December 20, 2016 @02:30PM (#13223096)
And you used Python as the example, probably has the easiest cryptography libraries to use.

C/C++, Ruby, JS, etc. are much, much worse in usability.

Re:Rolling Your Own Crypto by PopsiclePete on Tuesday December 20, 2016 @04:09PM (#13223997)
The Python eco-system is a ghetto. I don't understand why, it really is one of the most popular languages.