In my experience that's often due to major modes that try to format and syntax-highlight on the fly. Keep in mind that some forms of pasting are really equivalent (from Emacs' PoV) to a crapton of keystrokes being entered.

Anyway, the workaround I have for the occasional time I have huge things to paste is to switch the buffer to fundamental-mode before pasting, and then switching back.

As a curiosity, what kind of things are you pasting that are that big?