This is more of a challenge with newer, larger technologies (deep learning and large language models). The operation of older, smaller scale A.I. systems (rule induction, tree induction, [smaller] neural networks) are much more clear.

I've been working in A.I. and analytics for over 30 years and agree that throwing up one's hands, complaining about how "we can't understand" what A.I. is doing is a bit theatrical.