News for hackers. Stuff that matters.
Re:User interface design timeframes: from 0.1 seconds to 10 ... by fwlr on Tuesday March 07, 2023 @08:44AM (#35055224)
I remember reading this a long time ago - maybe even when it was first published? - and formulating some UX rules of thumb based on it.

0.1 seconds or less: the illusion of direct manipulation is maintained. “You feel like you directly made the thing happen”. There should be no loading indicators of any sort. Animation should be sparse and economical (at 60fps you have 6 frames to tell the whole story), or absent if possible (`duration: 0.3s` instantly destroys direct manipulation).

1 second or less: 2-way interaction with flow. “You feel like the computer made the thing happen”. There should be a loading indicator of the indeterminate activity-is-happening type (spinner, pulsing gray text skeleton, etc). A vague reassurance the computer is working on it is enough.

10 seconds or less: Loss of control. The computer has put you on hold, and you’re waiting for it to get back to you. There should be a progress bar, spinners won’t cut it. Users don’t like losing control - take this period seriously.

More than 10 seconds: Loss of user. Persist session data to localStorage so at least it’ll be there if they ever come back.

Re:User interface design timeframes: from 0.1 seconds to 10 ... by _dain_ on Tuesday March 07, 2023 @09:19AM (#35055607)
>0.1 seconds or less: the illusion of direct manipulation is maintained.

100 milliseconds is still an atrocious amount of latency in many contexts (typing for example). There were some Microsoft usability studies of tablets, people dragging their fingers on the touchscreen to draw things. They needed on the order of single-digit millisecond latency to make it feel immediate, because otherwise you could see the drawn brushstroke noticeably lagging your finger.

It's true that human reaction time is at that timescale, but that got thoughtlessly transmuted into "latency below 100ms doesn't matter", which doesn't follow at all. The hand-eye-brain system is a very complex prediction-feedback system, used to working in real life where there's usually zero real latency between what your fingers do and how the world reacts. Your brain is already doing the biological equivalent of lag compensation and rollback to account for how long nerve signals take to travel and how long muscles take to move and how fast its own neurons can process information. Adding more latency and jitter on top of that can only make the error rate worse.

And doubly so when there are different amounts of latency for the different sense modalities, e.g. visual vs audio.

See: https://danluu.com/keyboard-latency/#appendix-counter-argume...

Re:User interface design timeframes: from 0.1 seconds to 10 ... by moritonal on Tuesday March 07, 2023 @09:31AM (#35055730)
Yep, even worse in gaming where some user's are likely capable of noticing between 1-10ms latency differences. Not strictly, but enough to let a reviewer describe a game as sluggish as noted in today's PC Gamer article[1].

https://www.pcgamer.com/we-played-the-fps-with-unlimited-des...

Re:User interface design timeframes: from 0.1 seconds to 10 ... by fwlr on Tuesday March 07, 2023 @09:45AM (#35055879)
Yeah, these rules are scoped to “doing stuff by clicking on a web page”. They are rules for buttons, basically.

Typing absolutely needs to be “instantaneous”, i.e. it happens by the very next frame, so “< 16 milliseconds”. Same goes for mouse movement, the cursor has to be perfectly in sync with your hand movement. These are a step beyond “it feels like you made the thing happen”, they are more primitive, like maybe “it feels like you are the thing”. When you put touchscreen gestures in the same category as mouse cursor movement and typing, it seems obvious you would need single digit millisecond feedback - but it did take us some time to empirically discover that was the natural category for gestures.

Re:User interface design timeframes: from 0.1 seconds to 10 ... by _dain_ on Tuesday March 07, 2023 @10:24AM (#35056336)
>These are a step beyond “it feels like you made the thing happen”, they are more primitive, like maybe “it feels like you are the thing”.

That's a great way of putting it. I imagine VR developers have to deal with this.

Re:User interface design timeframes: from 0.1 seconds to 10 ... by fwlr on Tuesday March 07, 2023 @12:03PM (#35057802)
I vividly recall Carmack describing some of the lengths they had to go to to get VR respond to body movements. IIRC it was crucial to get the scene to move around exactly when your head moves around, or else your body would think you’re poisoned/dying or something. The PC would be polling head position and rendering at like 90fps and even that was too much latency so they had this insane hack where the rendered frame was sent down the wire to the headset, and the headset itself had some onboard processor that would skew and shift the frame according to the last data from the accelerometer literally right before putting the frame on screen.

https://developer.oculus.com/documentation/native/android/mo...

Re:User interface design timeframes: from 0.1 seconds to 10 ... by Ajedi32 on Tuesday March 07, 2023 @01:10PM (#35058805)
> the headset itself had some onboard processor that would skew and shift the frame according to the last data from the accelerometer literally right before putting the frame on screen

FWIW that's only needed when you don't have enough bandwidth to send an uncompressed video stream to the headset and need a decompression step that introduces additional latency (e.g. you're sending the video stream over 6 GHz Wi-Fi or USB instead of DisplayPort).

Tethered headsets do use a similar technique in situations where the game can't keep up with rendering at 90fps but that's done on the computer's GPU, not on the headset.

Re:User interface design timeframes: from 0.1 seconds to 10 ... by fwlr on Tuesday March 07, 2023 @01:34PM (#35059192)
Yeah, on re-reading I don’t know where I got the idea the headset was doing on-board processing. I recalled that rendering was done using positional data and they had to add a compositor post processing step afterwards using newer position data, but I hallucinated that the need for low latency was so extreme that it had to be done on the headset.
Re:User interface design timeframes: from 0.1 seconds to 10 ... by Ajedi32 on Tuesday March 07, 2023 @02:31PM (#35060018)
Again, it does happen on the headset, but that's only necessary in situations where there's not enough bandwidth to send an uncompressed video stream.

See Oculus Link over USB, for example: https://developer.oculus.com/blog/how-does-oculus-link-work-... The diagram there shows Timewarp running on both the PC and the headset side.

In headsets like the Rift or Valve Index that send video streams over HDMI or DisplayPort that extra step isn't necessary; everything happens on the PC's GPU.

Re:User interface design timeframes: from 0.1 seconds to 10 ... by fwlr on Tuesday March 07, 2023 @05:46PM (#35062391)
Ah, ok, that’s how I ended up with the original mistaken belief. I remembered “Timewarp is necessary” and “Timewarp can happen on the headset” and conflated them into “Timewarp necessarily has to happen on the headset”.
Re:User interface design timeframes: from 0.1 seconds to 10 ... by rented_mule on Tuesday March 07, 2023 @03:26PM (#35060701)
When building tools for developers and data scientist to iterate (any tool used in an iterative creative process like writing code, building ML models, etc.), I have similar breakdowns. This might be a build process (i.e., Makefiles and such). Or processing large amounts of data for analysis, simulation, model training, etc.

4+ hours: At most one iteration per work day. Because of the wait times, users will try to fit as many changes into each iteration as possible. Users just want to find something that works because trying alternatives can cost days, making results less optimal. Users will typically run jobs over night so they can work on the next iteration all day.

1 - 3 hours: 1 - 3 iterations per day. Users will do some exploration on the most important components. Context switching is a large cost because they have to do something else while it's running.

10 - 60 minutes: Still limited to a handful of iterations per day, largely because users don't want to do more context switches than that in a day. The run will finish sooner, but user won't be ready to switch back. Especially true later in the day as people get tired.

30 seconds - 10 minutes: Context switches can be simpler because they can be small tasks like catching up on email. Exploring different options is starting to pay real benefit here.

< 30 seconds: Context switches are now minimized, keeping the user in flow. Feels more like interactive exploration rather than idea -> do something else -> remember idea -> evaluate. Users will try lots of different options, often yielding much more optimal results.

Doing a bunch of work to get things from 5 hours to 4 hours is not of much use. Getting things from 10 minutes to 10 seconds greatly multiplies the value that others can create.

Re:User interface design timeframes: from 0.1 seconds to 10 ... by fwlr on Tuesday March 07, 2023 @05:37PM (#35062303)
Yes! 5 hours -> 4 hours is a quantitative performance increase; 10 minutes -> 10 seconds is a qualitative performance increase. Crossing from one time regime to another is a huge victory.