As I interpret GP, the claim is you can't describe something in sufficient detail to simulate it, then you don't actually understand it. You may have a higher-order model that generally holds, or holds given some constraints, but that's more of a "what" understanding rather than the higher-bar of "why".
It seems that they are saying that a simulation is required for proof. We write proofs for things all the time without exhaustively simulating the variants.
I never claimed that a simulation is required for proof, just that an unexpectedly broken (but correctly implemented) simulation demonstrates that the model is flawed.
Do you ensure this by simulating it?
Take for example cryptographic primitives. We often rely on mathematical proofs of their various properties. Obviously there could be an error in those proofs in which case it is understood that the proof would no longer hold. But we double (and triple, and ...) check, and then we go ahead and use them on the assumption that they're correct.
I'm not the previous poster, but how about the Halting Problem? The defining feature is that you can't just simulate it with a Turing machine. Yet the proof is certainly understandable.
On the other hand, assuming no errors in implementation then a broken simulation which you had expected to work directly implies that your understanding is flawed.