I'm an organist. The organ console is often placed quite some distance from the pipes—especially in the instance of remote divisions such as antiphonal stops (placed at the back of the sanctuary). In addition to distance, the mechanism of the instrument itself introduces a bit of lag. It's something that you just get used to. Choral and orchestral musicians also often deal with these sorts of issues: sometimes they have to play entirely on sight (watching the conductor's baton) rather than listening to the (delayed) sound.

I used to play Dance Dance Revolution quite regularly. One of our controllers had a lag that made it unplayable for most people. But I found my brain adjusting to it extremely quickly, and I think it's because I was accustomed to doing that already during organ playing.