Optical digital means perfect isolation from the ground plane of the computer. It's that simple. The DAC can do whatever it needs to manage its own noise levels, but you're pretty much guaranteed a huge difference from entirely decoupling the DAC from the computer's ground plane. That electrical interference can do a surprisingly enormous amount of damage to the analog circuitry of the cheap DAC, which itself is probably not very resilient at rejecting any sort of electrical interference.
So it's the DACs fault, but maybe it's fair to say that it's harder to make a good DAC when USB is how the data is being delivered. I'm sure a high end USB DAC doesn't have this issue, but I'm pretty impressed with my cheap offbrand optical DAC that was $11. I'm guessing I would not be so impressed with the $11 USB version.
Optical has the benefit of always being electrically isolated.
You'd be converting from USB to optical, at which point you'd break the ground connection which would be where your hiss is coming from (assuming it's still noiseless when still being used with optical). Then, your concern would be jitter and whether the added conversion is adding lots of jitter to the equation. Your DAC might (or might not) be good at rejecting jitter noise. I've got a Lavry DA10 that's exceptionally good at rejecting jitter (in crystal mode), but that's mastering grade and maybe overkill for you.
It wouldn't add literal noise, but it's also possible for the USB connection to be more jittery than a different computer making the optical connection. That's partly hardware and partly software design (controlling how the data stream is buffered, and associated things that might slightly modulate the audio data clocking). So a change in computer feeding the DAC could also substantially affect the 'sound' of the DAC, as well as the noise issue you observed.
This isn't a huge issue for me, it's just an example of where I'd prefer to use optical. If it ends up bothering me such that I need to fix something, I'll 'fix' the motherboard.