You may only have dealt with your own devices. Yes, VGA devices were much harder to accidentally unplug or become loose than HDMI. But they were often far more solidly connected than e.g. the mount on a video card, or the attachment of a laptop motherboard to it's case. So you would find components which had been messed up internally by the use of a VGA connector as a leverage point.