Given the possibility of dopant-level backdoors (see https://sharps.org/wp-content/uploads/BECKER-CHES.pdf ), you can't trust a chip you didn't fab yourself even if it passes optical inspection.

But maybe a simple FPGA design would be resistant to this technique if the attacker didn't know the intended bitstream in advance.