Your brain is also probability based and you don't have an audit trail for your thoughts written to a black box.

So what is really needed is to add post-hoc rationalization engines to these self-driving cars to facilitate satisfying our sense of justice.

That was actually my first reaction to reading about Sussman's AI ideas in an article[0] that was on HN recently[1]. Propagators seems neat, but the quote:

"Sure, when you're running the program, that whole thing is a black box. So is your brain. But you can explain to me the reasoning of why you did something. At that point, being able to inspect the symbolic reasoning of the system is all you have."

made me immediately notice that there's a reason we have negative connotations with the world "rationalization" - because most of the time, post-fact rationalizations are bullshit and do not reveal one's actual thoughts at the time of the event, but what one wishes one thought at that time. Just bolting a "reasoning explainer" on top of a Bayesian inference system won't fare any better than human rationalization does.

[0] - http://dustycloud.org/blog/sussman-on-ai/

[1] - https://news.ycombinator.com/item?id=10388795

I think the auto/software makers should take responsibility for it, though, as some already have said they will. Just like when you have bad brakes in a car, it's the auto maker that is responsible not the supplier of that component. Also, they can't just say "it happens, what are you going to do?"