Self-driving is fundamentally unsafe unless you do a height profile of the road ahead. That's step one. Right now, LIDAR-based systems can do that reliably, while camera-based systems cannot.

If you have to recognize obstacles to avoid them, sometimes you'll miss one. Once you know where you physically can drive, you get to work on where you should drive. And what the other road users are doing. The proper role of machine learning here is to help classify other road users and predict their behavior.

"Big rise or drop in road ahead" plus "classifier has no idea what this is" has to mean "Stop".