You can plug in multiple GPUs on a computer. AWS has fleets with 4 and 8, see https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html for pricing.
For anything datacenter related, customers are very sensitive to price per performance. And datacenters are happy to oblige.
You're missing the point GP made though. He wants to replace one good card with 4 worse ones. It's not like that rack has 3 or 7 additional slots just unused. They're also already taken by the setup. And in the link you provided 5 out of 6 offerings are still Nvidia GPUs.
Of course data centers are happy to oblige to customer demands, but initial cost per GPU and performance per watt are certainly not the only relevant factors.
That's not true from a hardware perspective either. You can't just plug in 4 worse cards in the same rack. The savings on the graphics cards become less significant if you need to double/quadruple all other hardware to increase the number of racks. A 1U blade can easily cost $10000 without a graphics card.