Yes, probably down to historical lack of contact. I don't think that tones are part of the reason. There has been somewhat more borrowing from Cantonese into English, e.g. ketchup, dimsum, kumquat, and cheongsam, and substantial borrowing from Chinese into Japanese and Korean. The tones are simply dropped.