10ms of added latency is very irritating - even adding 4ms feels clunky on guitar/headphones and that's typical for playing through a DAW and digital effects. As much as I like the iPad ... drawing on it is still like drawing underwater, and touch-to-sound latency is still too high in many cases. On the other hand... I can almost tolerate a bit of latency playing a weighted keyboard because I imagine that a hammer is hitting a string so I adjust my playing accordingly. The keyboard also seems to trigger before the note bottoms out, which helps a bit (except when you're playing very quietly.)

There was a terrible idea in the CRT era that when a user clicked on a menu you could do all sorts of computation to update menu contents and still have time to draw the menu before the next ~80Hz refresh. Even if you missed a frame "humans couldn't notice it."

Unfortunately this failed horribly in practice because it frequently pushed things over the edge, resulting in two frames of latency (or many more if latency wasn't managed well) which made things feel sluggish and unresponsive.

Even in 2019, though we have 120+ Hz user interfaces, asynchronous refresh, multi-gigahertz processors, and insane GPUs, menus are still clunky. Every time I click on a menu in macOS (Safari, for example, which I am using as I write this) I can almost hear it go "kachunk." Not even 10ms. ;-( Maybe the delay is an intentional fade-in or ... something, but I still hate it.

Moral of the story: 10ms is terrible, and every bit of delay you add to a UI can push it (and your users) closer to the edge.

Not to be too down on one player, but Apple is especially bad for this. GTK+ is considered slow by many, but compared to Cocoa, it's an SR-71. Cocoa rarely ever completes a window resize fast enough to hit a 60Hz VSync, GTK+ at least usually keeps up with 144Hz (for me) if you have a good GPU, and 60Hz if you have anything faster than a mid-grade Sandybridge IGP. Depending on the application, Qt is also sometimes dramatically faster than GTK+ (though on Linux the controls are somewhat clunky, since they seem to be emulating some Windows control behaviours which did not stand the test of time).

Field musicians (marching band, drum & bugle corps, etc.) have a similar issue because of the sheer size of the "stage"; the distances between performers is large enough that the speed of sound actually causes notes to be off-beat to the audience if they're "on-beat" to the performer.

Instead, each performer has to basically ignore one's own ears (at least for rhythm; one should by all means use one's ears for matching pitch, though in theory a sufficiently-fast-moving performer might need to pitch up/down relative to what one hears to account for Doppler shifts) and watch someone up front (e.g. a drum major). Even in non-marching settings (e.g. your typical orchestra or symphonic band or even rock/jazz/etc. band), the percussion section is typically positioned rear-most on the stage for this exact reason (and in this case, it is okay to listen for the beat, since the sound of said beat would be coming from behind you).

So for an organ player (organist?), your best bet would be to place the drummer behind the organ, then give the organist a mirror to watch the drummer's sticks and keep time with the visual beat. That way, by the time the organ's sound actually comes out, the drummer's sound will have arrived and they'll be in sync (assuming you have the distance right between the drums and organ; this is something you'll probably have to tune for every stage, given air-pressure-related differences in the speed of sound).

Organs have gotten better and worse in this regard several times over the hundreds of years since they have been known.

Another option would be to place the organist next to the drummer, but keep the organ itself far enough in front of the drummer such that by the time the sound comes out it's in time with the drummer. I don't know if there are any organs that work this way, though (I'd imagine it'd have to be either mechanically complex or electronically controlled; the latter, though, gives an option for a synthesized "monitor" that the organist can hear, which should hopefully make the delayed sound of the actual organ less of a mindscrew).

This all assumes you have a large enough stage to be able to pull this off, and (in the case of fixed organs) have actually built the organ such that the sound does emerge from in front of the drums.

It's related to most software engineering not being engineering: http://tonsky.me/blog/disenchantment/

Which explains why unless I missed some new development, there's no way professional music can be played live on any Android based product. By music I mean music, not triggering prerecorded loops, a thing which could be done on a 50+ years old Mellotron as well. I'm 100% sure Java is to blame: low latency and virtual machines don't mix well.

I shouldn't but I will anyway: People who are 100% sure of things are generally wrong and this case is no exception.

I'm being facetious here but what if I told you that nearly all machines are virtual machines? Your CPU doesn't really execute instructions, it interprets them by first translating them to microcode.

There's nothing inherent to the idea of not coding directly on the lowest level possible which forces your programs to have bad performance. Zero-cost abstractions really do exist.

I mean, Erlang uses a virtual machine called BEAM, erlang programs sit at a remarkably high level of abstraction, and it's not like erlang systems are known for their chronically high latency.

The Android Runtime also compiles applications before running them [1]. The instructions being run are native code in elf files [2].

Certainly Android has had latency issues, and I'm no fan of Java, but that was more of an architectural thing than a Java-Is-Such-A-Bad-Tool-Lol thing.

[1] https://source.android.com/devices/tech/dalvik [2] https://stackoverflow.com/questions/26254538/difference-betw...

Well that’s not true.

However, the lack of a widely-supported low-latency audio stack is still a killer.

Then, in future far, far away we might get a touch input method that isn't too laggy for interaction with audio. Last year, the best we could get was around 30ms[2] on iSomething; and I hope that 120hz refresh rate becomes the standard (but even that is huuuuge compared to a MIDI controller).

[1]https://github.com/google/oboe

[2]https://www.reddit.com/r/Android/comments/7teiwe/lets_discus...

There are some very recent developments on that frontier[1]. Let's hope that the new Oboe API will result in usable music software on Android.

I have a DX7 emulator on Android that is pretty good latency-wise when used with a MIDI controller.

Sadly, touchscreen latency is still way too big for anything musical (and the new Oboe API doesn't address that).

[1]https://android-developers.googleblog.com/2018/10/introducin...

I'm a pianist who sometimes plays Pipe Organ, and you always have to "play ahead." It can take 250ms or more for some 16' or 32' pipes to start sounding.

When I've used Audacity to record multi-track, here's my experience: Over time very good players will notice 5ms or above of de-sync.

It's like jumping instantaneously 3.4m away or toward, each note the timing is off by 10ms.

You, however, must constantly make such adjustments or the entire musical endeavor may fall apart. :)

Yes, I don't even always need to look at the screen when I am typing fast. However, when I do, molasses UIs make me sad.

As an interesting aside, I notice that my brain tends to make interesting typos when I am typing without looking at the words that I type and get into a flow state when I am not thinking about the spelling of words, especially at high WPM.

By "interesting" I mean "I would have never consciously chose to spell that word like that" interesting. It shows that typing is happening on a different level than spelling; it's a more direct pipeline than writing with a pen where one has to think about shapes of letters. It sort of happens as an independent process, where <send a word to keyboard> happens without active thinking, and can have interesting "optimizations" along the way. I am looking at the monitor to make sure that these "optimizations" do not, in fact, occur.

To give an example of this, here's a snippet of what I type with my eyes closed:

"I'm a rocket ship on my way to mars, on a collision course. I'm a sattelite, I'm out of contrrol. "

So, I need to occasionally look at the screen to catch "contrrol" errors. And it's annoying to do so when the letters lag behind key presses :)

Our visual systems are not optimized to give accurate, realtime representation of geometry around us. They evolved to give us a somewhat abstract representation that is most useful.

There is a lot of post-processing involved between photons hitting your eye and your brain "seeing" things. In that post-processing, both geometry and time information are "corrected".

The most striking example of this is saccadic masking[1]. Basically, your eyeballs move all the time, but you will never see that when you are looking at yourself in the mirror. However, look at your eyeballs on your selfie camera, and ta-dah! They move!

That's because your eyes essentially block your visual processing pipeline while the eye is moving, and then backfill the missing frames.

That's why if you move your eyes around rapidly while looking at an analog clock, you will perceive some seconds to last longer than others. The eyes are lying about how long things take.

>Our human image processing only runs at around 30-60 FPS when idle, but can increase in stressful situations

In general, our vision doesn't really have a "frame rate", and depending on how you define that concept, our "frame rate" goes well into the hundreds of FPS[0].

But something like what you said is happening. Visual perception of things gets on-the-fly adjustments as necessary.

For example, baseball players see balls as being larger than you would see the same ball while being stationary at the same distance[2].

That's why, yes, we can get away with a lot of visual trickery, taking advantage of the visual post-processing pipeline (monitors giving the illusion of smooth motion is one example).

However, this all doesn't mean we won't feel latency when it's there.

As a good example, compare drawing a line with a pencil vs. drawing a line on a touchscreen device with a stylus. You will notice even the smallest latency. In fact, there is no device on the market where something simple as drawing a line following a stylus would happen fast enough for you to be fooled.

As long as there is any visible distance between the tip of the stylus and the end of the line appearing on the screen, you will see it (or feel it), even if takes a very small time to fill that gap.

Same with scroll latency: it's something you will not be able to see on a video recording of someone scrolling on a touchscreen, but you will absolutely feel the difference when you do it.

And all that said, our aural perception is more sensitive to timing, and we also do Fourier transforms on the fly with our ears (that's how we can tell chords apart!), amonf other things.

Our perception is an interesting thing.

[0]https://www.pcgamer.com/how-many-frames-per-second-can-the-h...

[1]https://en.wikipedia.org/wiki/Saccadic_masking

[2]https://www.thecut.com/2016/10/the-science-of-how-baseball-p...

I suppose following a metronome instead of the drum isn't practical/possible ?

Learning an organ means you need to learn to trust the instrument (not easy when it is a mechanical beast with many parts that can fail) and feel what the music will be not react to what it is.

What about organ players?