I agree with you in that I don't think the problem is the closed-source aspect. Closed source software can still be audited (with difficulty). The problem is that the source material for the hashes can't be audited, even when we know exactly how the system works.