Not unless you are only downloading pre-compiled binaries.

Some linux distributions, for instance, try very hard to only every download source from the internet, and compile it locally. This is very slow (some things can take a very long time to compile).

Gentoo is one such distribution.

What downloaded binaries? The question was, is it really useful to have the source and schematics for the chips on your hardware, if NSA (as described in the article) is modifying the hardware during shipping, so you don't really know if the chip you have does what it's supposed to.

Hardware based rootkits can do pretty much everything to an OS running on that, and be undetectable from that OS.