I did some research and it seems that this particular slowness is due to single line file and if there is some syntax highlighting used with vim/neovim, it reads the line completely to do it correctly.

VSCode reads only the visible content and does not load everything for that line. It tokenizes the first 20k chars of the line at maximum, defined by the "editor.maxTokenizationLineLength" setting.