But I could argue that central agency validation _is_ probabilistic, because there is a probability that the central agency can collapse, be overtaken, go rouge, etc.

And that that actually might be more probable than the blockchain 51% attack or split or whatever.

I don't know what the actual math here is, I think that the essential difference is that in case of proof-of-work such a probability is computable relatively easily, while we don't know how to measure the trustworthy-ness of a third party reliably (it depends on laws, military strength, level of corruption, etc, lots of variables)

Using your definition, all cryptography is also probabilistic. It's technically correct, ie. there's a risk that someone guesses your AES key, but it's practically irrelevant because the probabilities involved are so small (and key sizes can be adjusted to make guessing as improbable to succeed as we want).

This is not the aspect that is considered to be probablistic. The problem is that forks can happen by design -- blocks can be orphaned and the tree can be reorganized. So a transaction is only final so long as it is on the main chain; if the main chain switches to a forked chain that does not contain your transaction, then it is not settled; the resulting coins cannot be spent.

There are attack scenarios, but even without those it is possible for forks to occur; single-block forks are common (so-called "orphaned blocks") due to some combination of network latencies and chance. Longer forks may be possible if there are more dramatic breaks in network connectivity. The original whitepaper originally advocated waiting for six blocks before a transaction is considered "settled". That's probably high, but the guarantees are strictly probablistic; any software dealing with bitcoins has to be aware of the possibility of reorganizations.