You might think you're joking, but there are gamer cases with space for two PSUs, and motherboards which can control a secondary PSU (turning both PSUs on and off together). When using a computer built like that, you have two main plugs, and the second PSU (thus the second mains plug) is usually dedicated to the graphics card(s).
PSUs can be removed from the PDB and replaced and reconnected to a source of power without having to shut down the machine or even remove the case lid. You don't even need to slide the machine out of the rack if you can get to the rear.
Example:
https://www.fspgb.co.uk/_files/ugd/ea9ce5_d90a79af31f84cd59d...
Edit: For example, in a 3+1 redundant setting, 3 PSUs would be active and contributing toward 1/3 of the total load current each; 1 PSU would be in cold standby, ready to take over if 1 of the others fails or is taken offline.
The second one was turned on with a paperclip, obviously.
Turns out graphics cards and hard drives are completely fine with receiving power but no data link. They just sit there (sometimes with fans at max speed by default!) until the rest of the PC comes online.
This would still have the disadvantage that the PWROK (grey) cable from the second PSU would not be monitored by the motherboard, leaving the machine prone to partial reset quirks during brown-outs. Normally a motherboard will shut down when PWROK is deasserted, and refuse to come out of reset until it returns.
In modern(last 4 years approximately) GPUs, physical wiring distance is starting to contribute substantially to latency.
The problem you will encounter with pcie gen5 risers is signal integrity.
It's about 75-90% the speed of light, but even that's too slow.
Modern hardware components are getting to latencies in single digit nanoseconds. Light travels about 30cms in a nanosecond, so extending a pcie port to a different box is going to have a measurable difference.
Also, these were vital to systems like the MacPro Trashcan that had 0 PCIe slots. This system was a horrible system, and everyone I know that had one reverted back to their latest 2012 cheese grater systems with the chassis.
There was another guy I know that was building his own 3D render rig for his own home experimental use when those render engines started using GPUs. He built a 220v system that he'd unplug the dryer to use. It had way more GPU cards than he had slots for by using PCIe splitters. Again, these were not used to draw realtime graphics to a screen. They were solely compute nodes for the renderer. He was running circles around the CPU only render farm nodes.
People think that the PCIe lanes are the limiting factor, but again, that's just for getting the GPUs data back to the screen. As compute nodes, you do not need full lanes to get the benefits. But for doubting Thomas types like you, I'm sure my anecdotal isn't worth much
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQqLXew...