You are right, exactly this way.
Ah, so both the physics and the Random.org affect the outcome of the dice. So you can't actually control the outcome of the dice by swapping out Random.org with a non-random source because the physics are also a factor.
So it should be impossible to precisely throw a value, since you can't control how you're holding the dice in your hand.