News for hackers. Stuff that matters.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @02:17PM (#6195634)
This is a Bitcoin software implementation bug, and an illustration of why you should use your OS's CSPRNG (here, /dev/random) to the exclusion of any other RNG.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by cperciva on Sunday August 11, 2013 @02:52PM (#6195782)
you should use your OS's CSPRNG (here, /dev/random) to the exclusion of any other RNG

Bad advice. Use your OS's CSPRNG to get a seed, but work with your own PRNG (say, HMAC_DRBG) internally. Going to the OS every time you want a few bits is both very slow and makes it far easier for local attackers to see when you're using entropy.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @03:19PM (#6195896)
Strongest possible disagree. For instance, the Debian OpenSSL bug was an entirely unforced error stemming from applications that chose to use Debian OpenSSL's terrible CSPRNG on top of the OS's CSPRNG. The local attacker scenario you're talking about is a theoretical risk when there are attackers running code in the same OS as your CSPRNG; a far more likely concrete flaw is, for instance, a SecureRandom implementation that allows developers to specify insecure seed values.

I'll go to bat on this disagreement. Don't use application-layer CSPRNGs. Use the one the OS provides, to the exclusion of alternatives.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by cperciva on Sunday August 11, 2013 @11:11PM (#6197454)
Strongest possible disagree.

Ok, we disagree. It wouldn't be the first time. ;-)

the Debian OpenSSL bug was an entirely unforced error stemming from applications that chose to use Debian OpenSSL's terrible CSPRNG on top of the OS's CSPRNG

I think the biggest problem was that OpenSSL used OpenSSL's CSPRNG after Debian broke it. Having applications all open+read+close /dev/random is not going to prevent that.

a far more likely concrete flaw is, for instance, a SecureRandom implementation that allows developers to specify insecure seed values.

A SecureRandom implementation should not allow the application to provide seed values except as additional input. It should always seed itself from the operating system entropy source, with no option to disable that.

Aside from the issue of performance (which can be severe) and the timing information leakage to local spies, there's a very practical reason to avoid developers read from /dev/random:

  int fd;

  fd = open("/dev/random", O_RDONLY);
  read(fd, buf, buflen);
  close(fd);
I've seen this on a number of occasions, and it works perfectly fine... until your server gets busy, the kernel entropy pool runs down, and /dev/random returns a short read. At that point, all hell breaks loose. To me, this scenario alone is enough to tell developers to use a library's automatically seeded CSPRNG instead of reading from the kernel.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @11:44PM (#6197548)
Linux random/urandom is supposed to prevent short-reads for exactly this reason. You raise an interesting point, but I'm not persuaded that the risk of quietly failing to read from the random device is greater than the risks of adopting an entire new CSPRNG to sit on top of the OS's CSPRNG.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by cperciva on Monday August 12, 2013 @01:06AM (#6197741)
Linux random/urandom is supposed to prevent short-reads for exactly this reason.

Oh great, yet another way that Linux developers are going to write broken code because they think the whole unix world behaves like Linux...

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Monday August 12, 2013 @10:54AM (#6199518)
So then what you should do is write a "libpcap for randomness" that uses the best-practices method of getting randomness from the OS CSPRNG. On Linux it'll be just a couple lines; on BSD it'll be a few more lines for the "atomicio" equivalent read, and on WINAPI it'd set up and call CryptGenRandom.

There's value in having a uniform interface to all the different OS CSPRNGs.

What I'm saying isn't valuable is duplicative effort to build additional CSPRNG logic in the app layer. As we keep seeing, these app-layer CSPRNGs are additional single points of failure; they don't add defensive depth.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tedunangst on Monday August 12, 2013 @12:22PM (#6200018)
How is OpenSSL RAND_bytes not that uniform interface?
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Monday August 12, 2013 @12:35PM (#6200110)
OpenSSL provides its own CSPRNG. When that CSPRNG breaks, as it has in the past, no amount of careful validation of the OS CSPRNG helps you. The OpenSSL CSPRNG is thus a second single point of failure.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by fulafel on Monday August 12, 2013 @07:45AM (#6198683)
The random/urandom devices do originate from Linux, other implementations should be careful about not making incompatible changes.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Sunday August 11, 2013 @03:53PM (#6196071)
The Debian OpenSSL bug was a result of Debian patching OpenSSL's RNG to make it utterly broken. Unless you're going to claim Debian couldn't have patched an equivalent bug into the kernel, you can't generalize this as support for the claim that kernel RNGs are inherently superior to library RNGs.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @04:34PM (#6196223)
I'm saying the error was unforced because the effort the OpenSSL CSPRNG went through was duplicative.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Sunday August 11, 2013 @05:16PM (#6196367)
So your position is that Debian didn't go far enough?

They should have disabled OpenSSL's CSPRNG entirely and redirected its internal calls to perform IO operations on /dev/[u]random?

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @07:13PM (#6196734)
Modulo the fact that Debian shouldn't be mucking with the inner workings of packages in the first place, sure; stubbing out OpenSSL's CSPRNG with calls to the OS CSPRNG seems like a reasonable plan. It is, by the way, part of the NaCL design too.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Sunday August 11, 2013 @10:17PM (#6197317)
Last I heard, there were still some "security oriented" OSes using RC4 for their /dev/random.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @11:26PM (#6197507)
arc4random predates modern OS CSPRNGs.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Sunday August 11, 2013 @11:49PM (#6197565)
So does OpenSSL. I think it's just not reasonable to expect a widely-portable crypto primitive and protocol library to judge the quality of the host /dev/[u]random and delegate to it. Of course, they seed from the host device and try to estimate the entropy it provides.

Even that OS with the biased /dev/[u]random using RC4 because it was fast would keep a separate RC4 state in each process' libc to avoid the overhead of kernel calls.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @11:54PM (#6197575)
So then, you think NaCL's design decision to rely on /dev/urandom was a mistake?
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Monday August 12, 2013 @12:15AM (#6197633)
I think if NaCL ever gets popular that design decision will bite somebody, somewhere, on some OS.

Our crypto libraries need to provide their own belt to wear with the OS-provided suspenders. Yes, I know about potential fork() bugs, but our modern VM-crazed deployments introduce that possibility at kernel level too. We need our good crypto implementers shipping good lightweight CSPRNGs, rather than the kernel developers who try to compensate with 6000 bit entropy pools.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Monday August 12, 2013 @12:22AM (#6197649)
If some OS screws up urandom so much that it bites someone, it'll bite a huge number of different apps, and not just the ones that use NaCL. Like it or not, urandom has to work; all sorts of things depend on it. The urandom ship has sailed.

Meanwhile, better to have just one codebase to review, rather than 100 crappy ones, like the SecureRandom from Harmony.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Monday August 12, 2013 @12:37AM (#6197682)
OK, so NaCL will not be guilty and they will be in good company.

I'm sure these principles of minimal redundancy will be of great comfort to those users who get their private keys exposed.

Do you see the paradox here? Find me these developers who are undoubtedly competent to implement their own CSPRNG, but prefer not to. These are the folks I want implementing the CSPRNG that will be generating my keys, rather than kernel developers who find entropy pools exciting.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Monday August 12, 2013 @12:41AM (#6197688)
You see my point, right? A lot of your security already depends on urandom working. Everyone knows this, and a lot of effort has gone into validating the kernel random driver (there have been some pretty good systems security papers on it). Given that you already depend on urandom, all a new CSPRNG gets most developers is a second single point of failure.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Monday August 12, 2013 @12:56AM (#6197720)
I get your point and I agree it's at least not a wrong way to look at it. But you say "the kernel random driver" as if there were only one for a crypto app developer to worry about.

We have already seen a huge number of bad keys generated. Is it that obvious at this point that kernel (and embedded system) developers are so much better at this than OpenSSL?

Is it possible that you just tend to look at more broken library and app crypto code during the course of your work? If you worked primarily with broken kernel crypto code, would you perhaps prefer (for your own use) a CSPRNG in a library written by your favorite experts?

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Monday August 12, 2013 @12:38PM (#6200132)
It is true that we see a lot of broken randomness code, and it is also true that we pay a lot of attention to failures of different CSPRNGs, but my point of view on this is also influenced by things like the design paper Daniel Bernstein wrote for Nacl.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by marshray on Monday August 12, 2013 @04:19PM (#6201502)
I'll wager your team has reviewed more crypto code, and more recently, than DJB.

Meh, this conversation needed to be over

      do random_beverage(); while (self->is_conscious());
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Monday August 12, 2013 @06:24PM (#6202576)
Tell you what: I will ask DJB tonight and see what he says.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tedunangst on Sunday August 11, 2013 @05:40PM (#6196429)
This seems the complete opposite of your usual advice to use a library. You want every app developer to memorize the intricacies of which of random or urandom or srandom they should use on which OS and whether they need to worry about blocking or short reads and whether the entropy returned is good, or really good, or just good enough? You don't think people are going to dick that up? I'd trust the OpenSSL devs to figure this mess out before John Q. Random developer. Any day.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @07:12PM (#6196727)
The OS already provides a library for securely generating random numbers. On Unix, that library is called "the random driver". Its interface is file-descriptor based.

You'd rather not be in a position to care about cryptographic randomness regardless. So, by all means, use high-level crypto libraries. But those libraries should also be using the random driver, instead of trying to bolt their own CSPRNG on top of it (or, god forbid, trying to avoid the random driver altogether).

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by andrewcooke on Sunday August 11, 2013 @05:02PM (#6196326)
"This is a Bitcoin software implementation bug,"

where are you getting your info from?

as far as i can tell previous code was buggy, and android 4.2 "fixed" things by making it impossible (well...) to screw up the PRNG (setSeed in OpenSSL augments state, previously with BouncyCastle it replaced state, afaict).

but that doesn't explain why current software has problems (not the kind of problems that should make it insecure - they may have problems with being unable to recreate keys if they were using seeded PRNG output as keys(!), but 4.2's changes should just screw them completely, rather than make things insecure), or why the article link blames the platform libraries.

so why are you saying this is a problem now in bitcoin library code? is there another link somewhere i've missed? (i agree it's suspicious that everything is bitcoin-related, but i can't see any certain evidence...)

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @07:59PM (#6196933)
You might be right! I'd be surprised if there was an OS or even framework-level CPSRNG bug in Android, but it's possible, and the obvious bug Patrick mentioned downthread is nowhere to be seen in bitcoinj or SpongyCastle.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by gwillen on Sunday August 11, 2013 @02:27PM (#6195671)
As far as I know that is NOT correct. The wallets in question were all -- to my knowledge -- using the Android platform-provided Java SecureRandom generator.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by patio11 on Sunday August 11, 2013 @02:38PM (#6195726)
As tptacek has mentioned many, many times, SecureRandom is one of those things which is very secure if you understand exactly what it is doing and do not shoot yourself in the foot.

One easy way to shoot yourself in the foot with SecureRandom is to use seed values from a source with low entropy.

http://developer.android.com/reference/java/security/SecureR...

If one were to copy/paste the sort of code samples which show up when one Googles for [Android SecureRandom], one would more than likely call setSeed, possibly in such a fashion as to duplicate seeds and, predictably, make the output of the random number generator deterministic.

I will leave it to one's individual judgement as to whether "It is unlikely a serious developer would copy/paste in crypto code into a project that has real money on the line" is descriptive of the prevalent standard of care in engineering in the Bitcoin community.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @02:40PM (#6195737)
They were explicitly seeding SecureRandom?
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by glurgh on Sunday August 11, 2013 @03:01PM (#6195821)
Not in the bitcoinj library they all appear to be using, no. That, in turn, uses bouncycastle. Bitcoinj is not explicitly seeding SecureRandom, I'm reasonably sure.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @03:20PM (#6195899)
I couldn't find a line of code in bitcoinj that explicitly seeded SecureRandom, or used the explicit-seed constructor.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by glurgh on Sunday August 11, 2013 @03:51PM (#6196066)
Neither could I. Maybe it is SecureRandom itself but then that would be a bigger deal than some broken BitCoin wallets, one would think.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by patio11 on Sunday August 11, 2013 @02:54PM (#6195794)
Don't know. This just seemed like the high-percentage guess.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by Dylan16807 on Sunday August 11, 2013 @02:41PM (#6195738)
If that's the problem, it does lead to a beautiful solution of fixing bugs by removing code entirely.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @02:35PM (#6195707)
If the story here is that Java SecureRandom on Android is bad enough to break DSA, the headline on this story is wrong; it should be something more like "Android Doomed".

Can you provide a link to something corroborating this?

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by gwillen on Sunday August 11, 2013 @03:31PM (#6195961)
Ok, more source diving. Bitcoin Wallet calls into bitcoinj java/com/google/bitcoin/core/Wallet.java sendCoinsOffline(...), which calls completeTx(...), which calls core/Transaction.java signInputs(...), which calls calculateSignature(...), which calls core/ECKey.java sign(...), which performs:

        ECDSASigner signer = new ECDSASigner();
        ECPrivateKeyParameters privKey = new ECPrivateKeyParameters(privateKeyForSigning, ecParams);
        signer.init(true, privKey);
        BigInteger[] sigs = signer.generateSignature(input.getBytes());
        return new ECDSASignature(sigs[0], sigs[1]);
This _appears_ to be a correct invocation of spongycastle-formerly-bouncycastle, initializing the signer with an instance of ECPrivateKeyParametes, and NOT an instance of ParametersWithRandom, so that, on org/spongycastle/crypto/signers/ECDSASigner.java line 41, we call SecureRandom() with no arguments.

I don't see spongycastle ever calling setSeed, and I don't see it ever leaking its SecureRandom instance, so unless calling setSeed on ANY SecureRandom instance is a problem, this _looks_ like a correct usage.

Also, I now remember how much I hate reading Java.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by gwillen on Sunday August 11, 2013 @02:53PM (#6195787)
I have done a bit of source diving in response to your question, and the answer is that all I can provide is the bitcoiners' announcement:

https://bitcointalk.org/index.php?topic=271831.0

They claim the problem lies with 'a component of Android'. One of them told me that the solution was to switch from using SecureRandom to reading /dev/urandom directly. The actual source changes appear not to be public, and he wouldn't tell me details about the issue.

I tried to find the usages of SecureRandom in a couple of the apps that are supposed to be affected, but a cursory search didn't turn up much. I suspect it's inside some library that I don't know to look inside. The question, I guess, is whether all the affected apps share the same library or not -- I don't _think_ so, but if they do it would dramatically reduce my confidence that the issue lies with the Android platform, as claimed.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by glurgh on Sunday August 11, 2013 @03:04PM (#6195827)
That's the lib, if you're bored enough to keep diving

https://code.google.com/p/bitcoinj/

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by gwillen on Sunday August 11, 2013 @03:15PM (#6195873)
OK, I dove a little more. The actual signing happens in org.spongycastle.crypto.signers.ECDSASigner . I haven't dug into _that_ yet.

But one thing I'm wondering is: would any setSeed on any SecureRandom instance cause a potential problem? Or just on the same instance being used for signing? In other words, do I only need to check that spongycastle -- which I presume is a fork of bouncycastle -- handles things reasonably? Or could any other messing about with SecureRandom mess it up?

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by benmmurphy on Sunday August 11, 2013 @08:47PM (#6197069)
the only other thing i see is the openssl securerandom code doesn't properly check the return value. it is possible for RAND_bytes to fail and for the securerandom code to not throw an exception. i think this is quite unlikely to happen. so basically:

1) reading from /dev/urandom takes more than 10ms causing an entropy error

2) a malloc fails causing the error to be saved to the fallback ERR_STATE instead of the thread local ERR_STATE

3) java sees failing code and tries to read the last error but now malloc succeeds so it reads it from its local thread ERR_STATE which succeeds or some other thread has clobbered the fallback ERR_STATE

4) java doesn't throw an exception because it sees no error

the other possibility is the openssl random state is shared between processes due to forking. i hear there is a process called the zygote that warms up the vm and forks to create apps. if it initialises openssl then it is possible child processes could get the same random state.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by robertotyley on Monday August 12, 2013 @11:06AM (#6199578)
SpongyCastle is a repackage of BouncyCastle specifically for the Android platform, an unfortunate necessity due to Android shipping with a cut-down version of Bouncy Castle (see https://code.google.com/p/android/issues/detail?id=3280 & http://rtyley.github.io/spongycastle/ ).

...I'm the packager of SpongyCastle (a task a well-trained monkey could do, I don't alter the code other than package-renaming it), and I confess some relief that the issue is with Android's in-built SecureRandom class, rather than anything in (Bouncy|Spongy)Castle.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by benmmurphy on Sunday August 11, 2013 @06:09PM (#6196535)
if they are using SecureRandom backed by openssl then i think any instance is a problem. However, the 'seed' is used to augment the state and not override it. I'm not sure if it is possible or how much data would be required to get predictable output from SecureRandom by calling setSeed.

http://www.openssl.org/docs/crypto/RAND_add.html

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @03:25PM (#6195926)
Not seeing any relevant code that sets seeds in SpongyCastle, either.
Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by gwillen on Sunday August 11, 2013 @02:59PM (#6195808)
Actually, someone elsewhere in the thread has found, and linked, the bugfix commit, in another Android bitcoin wallet:

https://code.google.com/p/bitcoin-wallet/source/detail?name=...

So literally they're just ripping out SecureRandom and replacing it with their own stub that reads /dev/urandom.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by VMG on Sunday August 11, 2013 @02:43PM (#6195750)
Well there is this paper: http://www.scribd.com/doc/131955288/Randomly-Failed-The-Stat...

Quote:

> Apache Harmony revealed multiple weaknesses caused by implementation bugs. As a part of Android a plethora of cryptographic functions [17] rely on this PRNG. One of the bugs addresses directly the Android platform, where as the second one only targets Apache Harmony.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by tptacek on Sunday August 11, 2013 @02:54PM (#6195791)
The bug described here isn't sufficient to generate repeated DSA k-values, even if it is still in the code†, which is unlikely. It seems more likely that the application code used SecureRandom insecurely, as described upthread.

† It's virtually certain not to be in the code, since Android's CSPRNG is based on OpenSSL now, not Harmony's built-in CSPRNG.

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by makomk on Sunday August 11, 2013 @04:07PM (#6196139)
According to Mike Hearn's comment over at the Bitcoin forums[1]:

> You should not assume that using OpenSSL directly is safe. On Jellybean+ the SecureRandom provider is just a shim over the OpenSSL RAND_* functions and Jellybean+ is also affected. However TLS connections are OK.

> I realise there's going to be a lot of questions about what exactly is going on here, but I'm not on the Android team and can't talk on their behalf. It's up to them to document exactly how the RNG is broken. Suffice it to say, if you aren't sure if you're affected, you probably are but you are welcome to send me a private message detailing what you're doing and I'll let you know.

[1] https://bitcointalk.org/index.php?topic=271486.msg2911376#ms...

Re:Android RNG Weakness Renders Bitcoin Wallets Insecure by benmmurphy on Sunday August 11, 2013 @05:53PM (#6196477)
that is kind of scary. i had a look over the code and the problem seems not obvious. the only issue i saw was multiple instances of SecureRandom share the same state which could cause some problems if you do [1]:

x = new SecureRandom();

y = new SecureRandom();

y.setSeed(predictable);

x.generateBytes(zz);

and also i think the seed is thread local so you can have:

x = new SecureRandom();

x.setSeed(xxx);

then on another thread x.generateBytes()

won't do what you think it will do. i don't think this would cause dupes because i assume apps are isolated from each other and these apps aren't calling setSeed...

[1] this is assuming android people haven't patched openssl to have different behaviour and also setSeed might be safe in openssl because it uses the value to augment and not replace the state.