https://jsfiddle.net/zs8bncxj/1/
I can still get it right pretty much 100% of the time, but the difference does feel a lot more subtle than what I was seeing in iTerm.
(If you change the delay, you need to hit randomize to make it take.)
Edit: Also we should keep in mind that we're not testing the latency we set in either test. We're testing that delay plus the delay from I/O. So 100ms might be indistinguishable. But by the time we pile everything on, we're getting closer to 200.
The result is the same as far as UI design is concerned; don't assume that you can get away with 100ms; most of that leeway has already been wasted by the OS and hardware.
Personally I can distinguish every time at 0.06.
I find it difficult to believe the rest of the hardware loop has 40ms latency, it would be difficult for smooth rendering to occur if it did.
I suspect that part of this is 'training' yourself but also what the researchers meant. They may very well have meant that above 100ms is jarring and noticed as a delay but below 100ms is not seen as a delay rather than truly being unable to notice in an A/B test.
I suspect this test would be more difficult if it sometimes did A/A and B/B.
Why would I/O latency have any effect on the smoothness of rendering? Are you sure you aren't thinking of throughput?
Remember: TFA just told us that many keyboards on their own add more than 40ms of latency. I definitely wouldn't have guessed that, so I'm very reluctant to entertain any certainty about the rest of the system having low latency.
I can get down to 24ms but no less... the weird thing is that 24ms is still completely obvious to me (clearly shorter but obvious in comparison to no delay), with a single test I can see which variable has the delay every time, but 1ms less and I can't... which makes me suspect it's being quantised due to any of the various things in between sleep and the output, display, driver, X, terminal emulator, CPU etc.
With that it's actually possible that my 24ms is larger than 24ms and is also being quantised to a larger duration (but not larger than 100 for sure).
It would be interesting to be able to test with some dedicated hardware.
It's not quite a keyboard delay test but more of a general human reaction time test.
The best I can do is 160ms after 5 tries on a 2560x1440 60hz IPS panel.
I'm using Alacritty, which is supposed to be really fast and everything!
Try it out.
If you want to blind A/B test yourself, run this:
Adjust DELAY as you want, and then use test_a and test_b and see if you can guess which is which, then run alias to see for sure.