The problem is secrets get massively abused on every level. From governments classifying evil to parents teaching their children to keep their abuse secret. While there are benefits for privacy, at least for the government, they do not outweigh the costs.