No, and that's the big problem.

You can't even tell by grinding the layers off one by one and inspecting them with a microscope. There was a paper published recently showing how to backdoor a random number generator by fiddling with the dopants. No visual difference at any magnification level. I suppose you could test for it with an electron microscope but you'd have to know what you're looking for and where to look.