it affects all loads, but for sgx the attacker scenario is that you have compromised the OS, which makes it significantly easier to create the conditions (faults/microcode assists) that cause the LVI than if you were just another userspace program (but it's not impossible there either)