We don't know what each connection means, what information is encoded in each weight. We don't know how it would behave differently if each of the million or trillion weights was changed.

Compare this to dictionaey, where it's obvious what information is on each page and each line.