Historically, observation and even usage precede understanding. Like planetary motion.

NNs (don't call it "AI", it's just one approach) are simply in a pre-scientific phase.

As are our own brains.