Does the preview not open up? I run Windows 11 but I'm so old school that I don't group taskbar buttons and you drag a file to the taskbar it will bring the app into focus and I drop it on that app.

But I noticed that I don't get the preview when dragging over the button but that might also have something to do with my particular setup.