This is the entire concern (in my opinion and in my reading of Tim Cook's opinion). If the government can force Apple to backdoor this one iPhone (because terrorist), then they can force Apple to backdoor any iPhone for any person given a valid warrant, subpoena or otherwise granted power. Once the flood gates open...
Imagine this scenario:
1.) Apple creates the custom iOS build for the FBI to use to decrypt this iPhone.
2.) China hacks into either Apple or the FBI and downloads this build. (We know they have the capability, because it's already happened. [1])
3.) A visiting U.S. diplomat, politician, or military officer has his iPhone pickpocketed while in China. (This also happens all the time.)
4.) The Chinese government uses this stolen software to brute-force the encryption on the device, finding access codes for classified U.S. military networks. (Because we know U.S. diplomats never use their personal email for state business [2], right?)
5.) Now a foreign power has access to all sorts of state military secrets.
The problem with backdoors is they let anyone in. Right now, there's a modicum of security for Apple devices because knowledge of how you would bypass the device encryption is locked up in the heads of several engineers there. The FBI is asking Apple to commit it to source code. Source code can be stolen, very easily. Tim Cook's open letter is making the point that once this software exists, there is no guarantee that it will stay only in the hands of the FBI.
[1] https://en.wikipedia.org/wiki/Operation_Aurora
[2] http://graphics.wsj.com/hillary-clinton-email-documents/
WARNING — THIS Apple Engineer IS CLASSIFIED AS A MUNITION --rsa--------------------------------8<------------------------------------- #!/usr/local/bin/human -s-- -export-a-crypto-system-sig -RSA-in-3-lines-HUMAN ($k,$n)=@ARGV;$m=unpack(H.$w,$m."\0"x$w),$_=`echo "16do$w 2+4Oi0$d-^1[d2% Sa2/d0<X+dLa1=z\U$n%0]SX$k"[$m]\EszlXx++p|dc`,s/^.|\W//g,print pack('H' ,$_)while read(STDIN,$m,($w=2*$d-1+length($n||die"$0 [-d] k n\n")&~1)/2) -------------------------------------8<------------------------------------- TRY: echo squeamish ossifrage | rsa -e 3 7537d365 | rsa -d 4e243e33 7537d365 FEDERAL LAW PROHIBITS TRANSFER OF THIS APPLE ENGINEER TO FOREIGNERS
edit: self-answer: the following post seems to have an answer: https://news.ycombinator.com/item?id=11115579, although it seems to describe a newer device than the one in the case; but I was interested in how such protection is possible at all, so that seems to answer it for me.
I strongly disagree. They are taking a stance in the debate about government mandated backdoors in software.
The order says that Apple's exploit should only work on this specific, already existing device.
That's a large part of the fuss!
So they would still get 10 bites at the cherry, and sure, on the tenth, they could depower the phone and prevent the wipe, but if each attempt is persistently stored before the password-check is carried out, depowering the phone wouldn't give them any more chances.
I'm ambivalent regarding Apple's stance. In principle they are doing the right thing, but in practice, it seems they may be kicking up a whole lot of fuss over a relatively minor issue (with the exception that providing an easy means to brute force a phone to the authorities sets a horrible precedent). As for creating a universal backdoor, it seems highly unlikely they couldn't produce a signed OS / coprocessor firmware image that wasn't locked to one of the various serial numbers associated with this particular device
edit: as mentioned below, this order entirely originates with Apple's use of DRM to prevent software modification. Had users actual control over the devices they own the FBI wouldn't need to request a signed firmware in the first place. Please think twice about what Apple might really be defending here before downvoting