I thought about this a lot when trying to come up with a very secure open source email service. Is there perhaps a way to show hashes for the coded binaries/etc that are used that could actually be trusted to be correct?

It just seems like such a chicken/egg problem. Where does the actual trust come from (like holy crap web certificates seems unbelievably broken).

Ultimately it seems like it's just impossible to be 100% for sure what is running on another persons server without having access to it. Which is unfortunate.