"I video record showing both the game controller and the screen with a 240 fps camera" https://superuser.com/q/419070/2269

So, it's total response times from key press on a game controller to screen that is important not individual components. Sure, 0.04 seconds on it's own is not a big deal, but when 5 or six things take 0.04 seconds you hit noticeable delays.