For a lot of people, we're well below the level of security where hardware security is relevant. The average person is still willing to run software that downloads/installs/runs malware on their machine (browsers with Javascript enabled). So in that sense, we've got a lot of work to do to get developers and consumers to practice better security at the software level before the hardware level is even a significant concern.
However, I do hope for a future where software security is both a fairly solved problem and the norm, and if that time comes, I want to be ready for the inevitable arms race when hackers move to hardware vulnerabilities.
Recognize that yes, there are always more layers, but each layer of vulnerability has decreased capabilities. It's a lot easier to read someone's hard drive using an installed app with full filesystem permissions than it is to read the same hard drive using their power supply.
Also recognize that these layers require more capability from your attacker. A Russian hacker can write an app that reads people's hard drives from anywhere in the world. The power grid simply lacks the networking infrastructure for long-distance remote access in many cases, requiring more physical locality.
I'm not saying this is pointless at all! Just that so far there seems to always be another layer that can be used to circumvent the protections.
I guess I'm more like saying we should not ever take security for granted. Instead, we should look at the whole system and try to find the easiest attack vectors. Which might not be purely software issues anymore.
For example, at some point we might need to consider the physical effects of the code being executed or how the physical environment can affect it.
When the software is sufficiently strong, the attacks will be directly or indirectly against the hardware.
Perhaps by something as simple as by a clever pattern of temp sensor reads causing your code to think temperature is in safe range for your application?
Or causing your code to execute extra multiplies to generate heat to hide a momentary temperature drop.
My point is even if you could 100% trust your supply chain and 100% validate your silicon matches your design, there are still residual issues.
For example, power supply glitch attacks [0] are nowadays well known, including techniques to make them sufficiently reliable.
Fair enough.
Has there to your knowledge ever been documented evidence that a mask had in fact been tampered with in a way that caused a system to be compromised?
I don't have any reference of a mask being modified to give to you, but it is so easy to do it that we don't actually need evidences to be worried.
If you think about it, just by changing implantation parameters on the transistors that form a ring oscillator for generating random numbers can bias it (this does not require not even modifying the mask).
RNGs are a weak spot. Thank you for that example.
It might be you shouldn't even trust the physical environment, even power supply can be used to do evil things. Or radiation, even ambient temperature.
Don't forget code itself can affect the environment in unobvious fashion.
EDIT: My point is that we need to be aware security will never be a solved problem. We can't also consider security as a software-only issue, but have to have a holistic viewpoint considering the whole system.
There's of course the point where risk mitigation is not worth the cost. That's another matter.