That is true and a good point. But when you think about it, with a large enough budget you could create hardware backdoors in microprocessors in the manufacturing plant without the chip designer ever knowing about it.

Then you're in a weird world, where a flagship smartphone has backdoors created by Samsung, Google, three different governments and the weird guy that worked with the chip design in team #7.