OK so a build machine is owned and we have a sort of consensus for trusted builders, and if there's a consensus mismatch we know something's up.

I suppose that's reasonable. Sounds like reproducible builds is a big step towards that, though clearly this requires quite a lot of infrastructure support beyond just that.