Well that clearly assumes that the user code is not utilizing most of the issue bandwidth or rather that the unutilized issue bandwidth is sufficient to run the RNG and that there are enough EUs left over to actually run it. In that case you do indeed get operations "for free" (in terms of wall-clock time), because your extra operations are exactly contained within the increased IPC.

It doesn't mean that it is infinitely fast. It does however mean that in this particular case it adds no latency for adding random number generation; an infinitely fast algorithm would also add no latency for random number generation. Doesn't mean one implies the other.