There will always be unanticipated circumstances that will foil the sensing systems. My go-to example is this situation where road re-paving artifacts and shadows combine to create a false lane marker. This vehicle's video-based system determined that the bright area between the shadow and lane pavement was the left lane marker and sounded a lane departure alert. Had the autonomous lane following feature been activate, it would have reacted and attempted to center the car on the false boundary:

http://imgur.com/a/KnsXU