What really got my goat about this article is that prior to the latest tested version of Gnome, the repaint rate was a fixed 40Hz! Whose decision was that?
That said, I can imagine more clever uses of displays might produce significantly faster displays.
The extra few bytes for Unicode characters outside of the usual ASCII range is effectively a rounding error compared with the bitmap data you’re copying around.
From a previous VTE weekly status message, leading up to the removal:
"I still expect more work to be done around frame scheduling so that we can remove the ~40fps cap that predates reliable access to vblank information." (from https://thisweek.gnome.org/posts/2023/10/twig-118/)
So as with a lot of technical debt, it's grounded in common sense, and long since past time it should have been removed. It just took someone looking to realise it was there and work to remove it.
Also, why do userland applications need to know the monitor refresh rate at all? Repaint when the OS asks you to. Linux is a huge pile of crap pretending to be a comprehensive OS.
You not understanding why something was done, much with your orignal comments about 40hz, doesn't actually mean that it is wrong, or stupid. It means that you should probably spend time learning why before making proclamations.
I can sort of see the argument for post actuation latency in some specific cases, but as a general rule, I'm struggling to come up with a reason to exclude delays due to physical design.
From what I understand, non-mechanical keyboards need the key to bottom out to actuate, whereas mechanical switches have a separate actuation point and do not need to be fully pressed down. In other words mechanical switches activate earlier and more easily. What you said seems to imply something else entirely.
(I would love to see scissors-action keys available to build custom keyboards, but I haven't seen any.)
It also completely ignores ergonomics. A capacitive-touch keyboard would have near-zero switch latency, but be slower to use in practice due to the lack of tactile feedback. And if we're going down this rabbit hole, shouldn't we also include finger travel time? Maybe a smartphone touch screen is actually the "best" keyboard!
I guess that is an open question - perhaps virtually all the variance in latency due to physical design is tied up with fundamental tradeoffs between feel, feedback, sound, and preference. If so - then sure: measuring the pre-activation latency is pointless. On the other hand, if there are design choices that meaningfully affect latency without meaningfully impacting other priorities, or even where gains in latency are perhaps more important than (hypothetically) small losses elsewhere - then measuring that would helpful.
I get the impression that we're still in the phase that this isn't actually a trivially solved problem; i.e. where at least having the data and only _then_ perhaps choosing how much we care (and how to interpret whatever patterns arise) is worth it.
Ideally of course we'd have both post-activation-only and physical-activation-included metrics, and we could compare.
I.e. I can want a particular feel to a keyboard which prioritizes comfort over optimizing travel distance independent of wanting the keyboard to have a low latency when it comes to sending the triggered signal. I can also type differently than the tester and that should change the travel times in comparisons, not the latentcies.
Musicians learn to lead the beat to account for the activation delay of their instrument - drummers start the downstroke before they want the drum to sound; guitarists fret the note before they strike a string… I don’t think keyboard latency would make rhythm games unplayable provided it’s consistent and the feedback is tangible enough for you to feel the delay in the interaction.
I'm a semi-professional keyboard player, and in the past I played with some setups that had a fair bit of latency - and you definitely learn to just play things ahead of when you expect to hear them, especially on a big stage (where there's significant audio lag just from the speed of sound). And some keyboard patches have a very long attack, so you might need to play an extra half beat early to hit the right beat along with the rest of the band.
If you watch an orchestra conductor, you may notice the arm movements don't match up with the sounds of the orchestra - the conductor literally leads the orchestra, directing parts before you hear them in the audience.
She's been learning Irish tin whistle for a few years, and is a big fan of the Dropkick Murphys and other celtic punk bands, along with 90s alternative bands lik Foo Fighters, Red Hot Chili Peppers, and Weezer. I've been learning guitar / bass / ukulele / mandolin, and it would be great fun if she can play drums and sing while I play something else....
I recall a guitarist friend who figured out their playing was going to hell trying to play to a track when their partner used the microwave. They were using an early (and probably cheap) wireless/cordless system and must have had interference.
On higher-end pianos (mostly grands), there is "double escapement" action, which allows much faster note repetition than without. I suspect the latency would be lower on such pianos.
> Musicians learn to lead the beat to account for the activation delay of their instrument
Yes, this is absolutely a thing! I play upright bass, and placement of bass tones with respect to drums can get very nuanced. Slightly ahead or on top of the beat? Slightly behind? Changing partway through the tune?
It's interesting to note also how small discrepancies in latency can interfere: a couple tens of milliseconds of additional latency from the usual — perhaps by standing 10-15 feet farther away than accustomed, or from using peripherals that introduce delay — can change the pocket of a performance.
Lots of people might not be good enough to care about missing 1-2 frames.
One could also pre-load the wasd keys.
Some of them will batch multiple key presses over a slower connection interval, maybe 60ms, then the radio to USB converter blasts them over together.
So you can still type fast, but you absolutely cannot play rhythm games.
It matters more in fighting games, where reaction time is crucial because you don't know in advance what your opponent is doing. Fighting game players are usually picky about their controller electronics for that reason, the net code of online games also gets a lot of attention.
In a rhythm game, you usually have 100s of ms to prepare you moves, even when the execution is much faster. It is a form of pipelining.
Don't even attempt to use a modern windows install, you end up feeling that it is actually broken!
I want an ARM laptop with expandable memory, user-replaceable battery, second SSD bay, and a well-supported GNU/Linux OS that has xfce as the UI - from the factory. That's the dream machine.
Although, mine is an x86-centric take. There are occasionally issues around getting the right kernel for ARM, right? So maybe it would be helpful there.
I've not tried an ARM laptop but this setup also works on RPi.
(if you only want keyboard then i3/Sway is great though, obviously)
> Still, it would be nice to return to the human resposiveness of machines 41+ years old.
A while ago someone posted a webpage where you could set an arbitrary latency for an input field, and while I don't know how accurate it was, I'm pretty sure I remember having to set it to 7 or 8ms for it to feel like xterm.
This is obviously an improvement, who cares if it is not perfect by your standards?
If the latency of those components varies wildly over the course of the test, it would introduce noise that reduces our ability to analyze the exact topic of the article - VTE's latency improvements.
Even if the latency of those components were perfectly consistent over the course of the author's test, then it wouldn't affect the results of the test in absolute terms, and the conclusion wouldn't change.
This exact situation is why differences in latency should never be expressed as percentages. There are several constants that can be normalized for a given sample set, but can't be normalized across an entire population of computer users. The author does a good job of avoiding that pitfall.
The Mutter thing is interesting. The author holds that constant, and GNOME sits on top of Mutter, so I think it's reasonable to assume we'd see the same absolute improvement in latency. GNOME may also introduce its own undesirable variance, just like keyboard latency. I'd be very curious to see if those guesses holds up.
I've actually been curious about getting a wireless keyboard recently, but wondered about how big of a latency impact there would be. Normally I use IDEs that definitely add a bit of sluggishness into the mix by themselves, but something compounded on top of that would probably make it way more annoying.
A quick search lead me to this site: https://www.rtings.com/keyboard
It does appear that they have a whole methodology for testing keyboards: https://www.rtings.com/keyboard/tests/latency
For what it's worth, it seems that well made keyboards don't have too much latency to them, even when wireless (though the ones that use Bluetooth are noticeably worse): https://www.rtings.com/keyboard/1-3-1/graph/23182/single-key...
Just found that interesting, felt like sharing. Might actually go for some wireless keyboard as my next one, if I find a good form factor. Unless that particular keyboard does something really well and most others just have way worse wireless hardware in them.
Blutooth is still crap though, avoid that if you care about latency. Or reliable connections.
Also be aware that despite being able, not all keyboards poll at 1KHz. My old wired keyboard polled at 66 Hz.
My current wireless keyboard has a single-key latency around the 1.7 ms range. Better than many wired ones!
https://www.youtube.com/watch?v=nbu3ySrRNVc explains it and has some statistics.
The real improvement wouldn't be reducing latency as much as allowing VRR signaling from windowed apps, it'd make the latency far more consistent.
Without it, you can race the scanout and minimum latency can get well under 1ms.
This was required for me to get a video stream down below 1 frame of latency.
No the minimum will be 0ms, since if the signal arrives just before the monitor refreshes then it doesn't need to wait. This is why people disable VSync/FPS caps in videogames - because rendering at higher than <60FPS> means that the latest frame is more up-to-date when the <60Hz> monitor refreshes.
The maximum monitor-induced latency would be 16.6ms. Which puts the average at 8.3ms. Again, not counting CPU/GPU latency.
33.3ms would be waiting about two frames, which makes no sense unless there's a rendering delay.
The changes described in the OP are unequivocally good. But the most promoted comment is obviously something snarky complaining about a tangential issue and not the actual thing the article’s about.
Yeah, one of those is that modern stacks often deliberately introduce additional latency in order to tackle some problems that Apple IIe did not really have to care about. Bringing power consumption down, less jitter, better hardware utilization and not having visual tearing tend to be much more beneficial to the modern user than minimizing input-to-display latency, so display pipelines that minimize latency are usually only used in a tiny minority of special use-cases, such as specific genres of video games.
It's like complaining about audio buffer latency when compared to driving a 1-bit beeper.
But wait! Not so fast (smile). This benchmark uses the compositor "raw Mutter 46.0", not GNOME Shell. Raw mutter is "a very bare-bones environment that is only really meant for testing."
In addition, this measurement is not end-to-end because it does not include keyboard latency. In this test, the "board sends a key press over USB (for example, Space)". Latencies just within a keyboard can go up to 60msec by themselves: https://danluu.com/keyboard-latency/
What are the true end-to-end numbers for the default configuration, which is the only situation and configuration that really matters? I wish the article had measured that. I suspect the numbers will be significantly worse.
I do congratulate the work of the GNOME team and the benchmarker here. Great job! But there are important unanswered questions.
Of course, the Apple //e used hardware acceleration and didn't deal with Unicode, so there are many differences. Still, it would be nice to return to the human resposiveness of machines 41+ years old.