That's actually some easy math, isn't it?

Well, I think that you can't predict if the driver of the other car will brake (or accelerate).

And even if: are its breaks strong enough to prevent it from crossing your way? (Or the other way round: What is likely its maximum acceleration?)

Please note that all of the above can happen even without any adversarial intentions.

The difficulty of the situation in general is why limitations should be placed on the system and a conservative approach taken in practice. Trying to circumvent difficulty with complexity is not an option for the first roll-out of a product.

Deploy self-driving on marked single-lane AI-only highways. Maybe let self-driving cars use bus lanes. But that's it for now, until we can certify and understand limitations of a system. It's only fair to the public to have slow rollouts that require mostly-attentive driving.