There is a lot that is not clear about the Anglo-Saxon migration to Britain. The obvious archaeological evidence suggests a very abrupt shock that sees Roman Britain almost completely replaced by Anglo-Saxon migration in maybe a generation or two. This has traditionally been interpreted as effectively a complete population replacement, although this has been challenged in more recent archaeology.

Probably, I would say, the big reason for the lack of Celtic borrowings into English is the fact that, to whatever degree there was admixture between Celtic and Anglo-Saxon populations, English would have been the prestige language, and the prestige language tends to be little affected by borrowings compared to the non-prestige language. You see this with French borrowings: French was the prestige language of England for about 400 years or so, so English starts borrowing all sorts of things from French, but not vice versa. The existence of the Danelaw region--a part of England that was primarily Danish rather than English even when largely ruled by the English--provided a substantial contact region for Old English/Old Norse borrowing, which was helped by the very similar nature of the two languages.