Or you are woken up by an airforce guard after your iCar attempts to drive onto the base. Car approaching a checkpoint + not responding to vocal commands + no apparent driver = not good things.

When self-driving cars become commonplace, they're going to learn very quickly to alert the driver that they're approaching the destination... particularly in these spots.

The base may not even be marked on the map. I came across an example [1] recently, just a blank space on Google maps.

[1] https://en.wikipedia.org/wiki/Aviano_Air_Base

On Bing Maps, it's visible in the satellite imagery but just fields in the birds view perspective.