Indeed that's what I conclude too. But if I have to configure, tbh I would much rather do it in Emacs lisp than editing JSON. The elisp is a very pleasant and full-fledged programming language that's well integrated and abstracted for configuration. It's also plaintext so can be version controlled like you said.
The "devcontainer" thing does sound like a very nice feature. I am intrigued about how it works though. One can always use volumes/bind mounts to edit source with host Emacs alright, but things like Language Server needs to know location of say 3rd party Python dependencies installed only in the container to provide autocompletion for them. Can VSCode really do completion when language server runs in host, but dependencies are only installed in the container? That would be a nice out of the box feature for sure.
It is definitely somewhere between architectural overkill and very useful.
But the cool thing is that you can override the configuration per workspace and that the config files are just JSON that can be checked into git. Another cool thing is that I can open a workspace in a "devcontainer", which means I don't have to install extensions and so on on the host.
What I really don't like is the terminal from VSCode. It's nothing compared to Alacritty.