P.S.: On reflection, I could probably have shortened all that by describing it as a "timing attack" [0] except in meat-space.

One mitigation might be to get the user to enter digits at a consistent pace, by forcing a delay between showing the random layout versus accepting a button press. There would need to be some penalty for early presses, to keep lazy users from just tapping the desired button repeatedly until it became active.