I'm pretty sure everyone (at least on M1 macs) are getting that 50-100ms delay. There's a perceptible lag vs Terminal.app - in the grand scheme of things it means nothing. But once you notice it... you can't unsee it!