Except if we are wrong in those other cases, we can usually use the legal system to help us recover our stolen property.
> So if you have the private key in your hand, and the address with your Bitcoin still has Bitcoin, while someone else could have your private key, why wouldn't they just transfer the money out?
Maybe they are waiting for you to deposit more bitcoin into that address, or maybe they are waiting to collect more private keys before exploiting them all at the same time, or maybe they... why speculate?
The idea that money can disappear without recourse because of one mistake in security or one highly motivated targeted attack is not, in my opinion, a desirable property of a currency.
Well, from a criminal's point of view, I don't see the upside in waiting. The owner of the coins could move them at any time, and then your loot is gone for good. I think it's safe to assume that if someone else has possession of your private keys, your addresses will be drained immediately.
So if you have the private key in your hand, and the address with your Bitcoin still has Bitcoin, while someone else could have your private key, why wouldn't they just transfer the money out?