I just flashed one of my keyboards with the "official" ZMK layout and it's so much better. I must have missed quite a few settings when I tried replicating it myself. I can't notice the delay at all.