Proving that the plans themselves aren't backdoored (your layout software was written by whom exactly?) and aquiring a microscope that can resolve 32nm features without using any integrated circuts (which, by paranoia, we know will identify images of backdoored ICs and replace them with images of non-backdoored ones) is left as a exercise for the reader.
I'm probably forgetting some other attack vectors here, actually.
I'm thinking of something like Vernor Vinge's "trusted computing" stuff in his SF book _Rainbows End_, where a flipflip took 20K gates to implement. You spend a lot of circuitry on attestation.
It's probably a circular problem (who will attest to the attestors? etc.), probably devolving into digital signatures of chip layers and stuff, taken before installing CPUs an peripherals... :-/
This, of course, doesn't make the backdoor a good idea, and back door or not, if someone does get the discrete log for the Dual_EC_DRBG constants, you are screwed.
If Edward Snowden was willing to give up his salary and liberty for what he thought was the moral thing to do then I think it's reasonable to assume there must be many more people within the NSA who are willing to leak secrets for money or loyalty to nation states, corporations or terrorist groups.
Some of those people will have given the Dual_EC_DRBG constants or applications that will crack cryptosystems using that PRNG to 'bad' actors.
Even if it was backdoored by the NSA, it would still be very useful for everyday threats. There are certain classes of attack that are difficult or impossible to prevent in software (think evil maid attack on full disk encryption)
The reality for hardware is you have to trust the guy who fabricated it, Unfortunately since you can't fab your own 32nm chips, we are unfortunately stuck trusting that neither the Chinese MSS or NSA has hacked them.