This post implies you can mark a canvas as "read frequently" to improve getImageData.

I'm excited to try it out, as I do a kind of hacky hit detection using a hidden canvas, And reading the pixel color is certainly the bottleneck there