This paradox is already evident in commercial flight: as automation has increased, flying has become safer, but pilots themselves understand less and less about the machines they operate, and when automation fails they are ill-prepared to get themselves out of messes.