Depending on the USB input device, that's not necessarily true at all. You could very easily find crappy USB input devices with greater latency than your ping to the news.ycombinator.com server, unless you're in Antarctica or something. There's at least one John Carmack twitter rant about ping to Europe being better than latency from some USB input device.

If you're a single router hop away from something the latency can be imperceptible. Whatever trouble is involved here isn't down to network latency.