Out of curiosity, do you (or any Android users reading this) find yourself doing this with any frequency?
Last example: Angry Birds
It wasn't in the Market at first. And many seem to do it since getjar.com was down some hours after Rovio announced via twitter that you could get it there.
I host pre-beta versions of my Apps on my own servers and give friends (and their friends) access to it. This would be impossible with iOS. I would have to know and register the hardware IDs of all devices in advance.
If you think this doesn't matter in practice, consider Apple vs Google Voice on the iPhone. Apple rejected (or "didn't approve") voice, leaving iPhone users with no way to access it. Users would be better off if Google could have hosted a Google Voice package external to the app market.
Swype keyboard is an example of an app that's not in the market but can be installed via APK. For most users, it's not a big deal, but I think that's mostly because the Market has far fewer rules than the AppStore, so even apps that require root access can be uploaded to the Market.
"Open" isn't very useful PR - which is what it basically feels like with Google at the moment - and I think it's fine Jobs is trying to highlight the advantages of the iOS platform.
37signals should move on from Political Communication 101 and avoid getting caught up in spin vs. content.
First and foremost, I don't have to jailbreak my phone to install apps not "approved" by Google/the carrier/whoever. I can download an APK from anywhere.
Secondly, the code is open. This does matter in practice, because there are people who can and do take advantage of this to create modified versions (e.g. Cyanogen). If Google decides not to take a particular direction, but enough people want it, it can happen.
Thirdly, it's not tied to one hardware manufacturer. Apple may make great hardware, but they attack a certain target market, which not everyone fits into.
Finally, although your argument about carriers has some truth, it misses a couple of points. It's not like iPhone users are exempt from all carrier restrictions - why can I tether via my Android phone yet iPhone-owning friends cannot? And there is nothing about Android that forces you to use a certain carrier (ironically, in the US, it's iPhone users that have no choice of carrier). I purchased my phone on T-Mobile in the UK, now I use it on another carrier in Australia, no problem.