I don't think I'm confusing them – but I do think that their "open-source"-ness is virtually worthless, as the only thing you're allowed to do is read the source yourself and hope that it's the same thing that you're getting from the official packages and servers.

That's a totally different aspect that reproducible builds are supposed to solve.

Your "problem" wouldn't be solved by allowing third-party clients. You still don't know what the server source is.