Have you tried the neovim plugin instead of the regular vim one? I find they have different trade offs, but the neovim one is actually using neovim inside a vscode window and requires a neovim install.

Particularly, I’ve found the undo functionality to be far better with the neovim one, it never screws up. Holding alt and moving a line up and down can sometimes accidentally move your cursor one line higher or lower though.