This is a good point, I'd want my car to be more like you than me, where I'm the guy impatiently going around people who hesitate for too long.

Just because it took it's time to navigate the problem it doesn't mean the decision was wrong. If you're in a rush you could easily take over the auto-drive feature. But otherwise let it auto drive in cautious mode for 99% of the time.