Optical digital means perfect isolation from the ground plane of the computer. It's that simple. The DAC can do whatever it needs to manage its own noise levels, but you're pretty much guaranteed a huge difference from entirely decoupling the DAC from the computer's ground plane. That electrical interference can do a surprisingly enormous amount of damage to the analog circuitry of the cheap DAC, which itself is probably not very resilient at rejecting any sort of electrical interference.
So it's the DACs fault, but maybe it's fair to say that it's harder to make a good DAC when USB is how the data is being delivered. I'm sure a high end USB DAC doesn't have this issue, but I'm pretty impressed with my cheap offbrand optical DAC that was $11. I'm guessing I would not be so impressed with the $11 USB version.
Optical has the benefit of always being electrically isolated.
You'd be converting from USB to optical, at which point you'd break the ground connection which would be where your hiss is coming from (assuming it's still noiseless when still being used with optical). Then, your concern would be jitter and whether the added conversion is adding lots of jitter to the equation. Your DAC might (or might not) be good at rejecting jitter noise. I've got a Lavry DA10 that's exceptionally good at rejecting jitter (in crystal mode), but that's mastering grade and maybe overkill for you.
It wouldn't add literal noise, but it's also possible for the USB connection to be more jittery than a different computer making the optical connection. That's partly hardware and partly software design (controlling how the data stream is buffered, and associated things that might slightly modulate the audio data clocking). So a change in computer feeding the DAC could also substantially affect the 'sound' of the DAC, as well as the noise issue you observed.
This isn't a huge issue for me, it's just an example of where I'd prefer to use optical. If it ends up bothering me such that I need to fix something, I'll 'fix' the motherboard.
If you used optical before, it can be reasonable to use USB->optical. A lot of USB audio interfaces already have this. I'm not making use of it, but the USB interface I have on my Mini has an optical out.
I have an external DAC/amp, and I had been using optical until I recently upgraded my workstation. Now I am using USB, because the new motherboard doesn't have optical; I assumed USB would be fine, so the lack of optical wasn't something I considered with the purchase.
Unfortunately there's an awful amount of hiss with USB on this DAC. I'm not knowledgeable enough to pinpoint what's causing it, but standard remedies I've found (eliminate ground loop, try ferrite cores, get a usb filter) have not made a difference. It could be the USB interface on the DAC side, I have no idea.
While I don't _need_ an external DAC, I like using one for a few reasons: mine has a low output impedance which is good for some headphones, a convenient switch to toggle between headphone and speaker output, and a big volume knob with really smooth action. I like it. The alternative is to use the motherboard's ports; this motherboard does jack sensing such that the rear line out gets disabled if something is plugged into the front, so switching between headphones and speakers involves plugging/unplugging the headphones all the time. I also don't like adjusting volume through a tray applet.