They already do. http://en.wikipedia.org/wiki/RdRand has been around on x86 for a while.
Unfortunately, they're essentially unauditable and sometimes frightfully broken, eg https://arstechnica.com/gadgets/2019/10/how-a-months-old-amd...
Fun times
So Linus is fine with it, and FreeBSD fed it into yarrow/fortuna which is the kernel boot time initialized entropy pool.
Putting it into a pool is fine. It can be XORed with other sources.
And if at boot time there's no stored entropy and no other sources, then it's better than nothing until more sources become available.
Open gate transistors, thermal effects, radiative effects...
(Not that there are not risks, and it is a mistake to assume on-die is not risky)