Some are self-executing, by which I mean that they can be enforced "by construction" - if you create an object that doesn't follow the rule, other people will know. Said another way, breaking the rule would also require breaking some crypto.
Some rules, though, are social. For example, why does everyone try to extend the longest branch in the block chain? Sure, the protocol says it's the rule, but why should that mean anything? People don't follow rules because they want to. They follow rules because it's in their enlightened self interest to do so. If you could make money by choosing a different rule, somebody would do that instead. So it must be that these rules get followed because it's in the interest of Bitcoin players to follow them. The natural follow-up question is whether these social/economic rules are stable. That is, why not some other solution? Why not only extend blocks whose (nonce % 0x0d) == 0?
Cryptographers use a very particular notion of security in which they like the security of their schemes to "reduce" to a well-understood assumption. That is, we prefer it you can prove something like "if you can break my system, then you can also solve problem X" where problem X is well-known and widely thought to be very hard. Then either I am forced to believe that your system is secure or that you have found an efficient way to solve problem X. And since solving problem X is unlikely, I should consider your system secure.
As I said, some parts of Bitcoin do reduce in this way to known cryptographic primitives (which in turn reduce to problems we believe are hard). But not all the parts.
You're taking a fundamentally confused position here. Consider: You have UberCoin— it is whatever spherical cow cryptocoin. EvilPal(Inc) then forks UberCoin and replaces its insides with a frontend to their EvilPal website (which then also copies in all the UberCoin ledger). By reduction, all digital currency systems are "social rules". It's not a useful description.
The longest chain rule is a rule that comes about for creating consensus in the system. If you change it in yours you will eventually break off into a private universe. The behavior actually does work by construction— but like my EvilPal example, that doesn't stop a stupid user from changing their own software to break off into another universe.
If you would like a longer formal analysis of the consensus mechanism in Bitcoin please see: https://socrates1024.s3.amazonaws.com/consensus.pdf
But its not set in stone and the lead developers do have a huge amount of control.
Also many disagreements in the forum about the max block size and blocking Satoshi Dice do worry me.
There's still time, but it really is something you don't want to mess up. https://bitcointalk.org/index.php?topic=157141.0
In the case of Bitcoin there's the 51% attack, which is pretty serious flaw even if to carry it out is unfeasable.
We don't know if NSA or GCHQ have broken bitcoin (although they've certainly investigated it); and we haven't seen much crypt-analysis on it; but we have seen very many people trying (sometimes successfully) to attack Bitcoin and the wallets and the exchanges.
There isn't anything magical about the number 51, the original Bitcoin paper gives a program that can be used to compute the success probability for an attacker with a given share of the computing power to reverse transactions to a given depth... At >=50% the probability becomes one but at that point only if the attack is of infinite duration. But, for example an attacker with 40% hashpower can reverse six blocks with a 50% success rate.
Any kind of consensus system is going to be vulnerable to a majority attack of some kind— otherwise you'd have "minority imposes a wrong outcome" attacks.
At least in Bitcoin the "consensus" only controls transaction ordering: a majority attacker can't inflate the supply or spend other people's coins... they can only DOS and remove transactions from the fixed consensus (and thus replace their own). Which is powerful but still limited.
This is interesting because if it wasn't secure I wonder why hundreds of millions of dollars have not been stolen yet? You could probably steal 100 million from some top addresses, move to MtGox and sell for a very decent rate before what you did became apparent (of course once what did became apparent Bitcoin would collapse forever)
Or if the addresses are secure and its something else that is not secure what is it? I am on the bitcointalk forum a lot and I have not read anything about the crypto being not secure.
Maybe I misunderstand what you mean by security?