Not everybody uses the PCI Express physical slot that you may be familiar with from standard PC motherboards. For example, see the OCP 3.0 card standard. They are designed logically and electrically for hot plug.

That said, hot plugging is definitely a "pets" thing and as for large scale data center computing, nobody cares. The machine fails as a unit and it's OK to take a machine offline to fix it.