What would you prefer it do? That "new layer" is how it asks whether you meant to paste the contents of your clipboard into an actual new layer or merge it into an existing one (anchor).

The relevant docs for the temporary floating layer that's created on paste are here: https://docs.gimp.org/en/gimp-selection-float.html