We have the same thing with LLMs. We don't know how they work. We don't understand them. We have only empirical, statistical indications, nothing is proven. We know how the small parts behave, we really understand those, and also the different algorithms. But they whys? No. We're getting better, but still far away.
As far as not knowing how it works, that becomes a bit of a philosophical problem. We know how LLMs work and their algorithms. I think we can always play up "we don't know" is disingenuous ways. How many people who made Apollo calculations could also write long proofs and theorems of the fundamental maths they were using from first principles? Not a lot. We're always standing on someone's shoulder. We'll never truly know everything or how exactly our tools work on an individual level. Maybe AI changes things because an innovation may not have a direct line to a Isaac Newton at its base, but I think that's a bit of a vague and poetic difference, and not a real one. Its still human achievement but via machinery.