You can (which is not, necessarily, to say do) know if the compiler is tweaked: http://www.dwheeler.com/trusting-trust
This only works if you are building things yourself or trust the group building things, of course, but it's way easier than audit by disassembling binaries.
How does this deal with [1]? Also, how do you know that your disassembler isn't compromised?
[1] http://programmers.stackexchange.com/questions/184874/is-ken...
Disassemblers produce assembly code, not the HLL code, so they are many orders of magnitude easier to write from the scratch than modern compilers. They typically expect human involvement as soon as there's non-trivial assembly-level engineered self-modifying code. Hopefully there's no much of such code in the results of the compilers we use.
Also if you check the whole discussion you'll see I already discussed Ken's work.
Ok, I appreciate this information (and I'm trying to follow the discussion but I didn't see you talking of Ken's work).
But I'm still curious; even though you can write the disassembler by hand, how can you be sure that you're compiling it with a non-compromised compiler? Or do you mean write it in e.g. ELF format directly (and that's assuming the OS isn't involved in filtering offending code, though it seems extraordinarily unlikely that the OS could be generally modified in such away without detection)?
The more general and diverse the tools you use, the less likely they are all compromised in the same way, and the more likely any compromise will show up in other contexts. Using tools at different meta-levels may also be worthwhile (machine-code vs. interpreter).
http://cm.bell-labs.com/who/ken/trust.html
For security conscious the prefect state is the OS which changes very, very slowly, fixing only security bugs and having binaries used by as many people as possible and which change so seldom that more people can even check them by disassembling them. You don't want to only check sources, you want to disassemble the binaries and decide if they match the sources.
And only then you want to be sure that all configurations are what they should be. Not easy at all.