Well, it's a Linux VM/container, so while you could do that, I don't see why you would, I think all a dev would want to do in a dev container would be command-line in nature.

It's not just a dev container, it's meant to be your workstation, with the code files being stored in it and not on your personal machine. So if I want to install IntelliJ, Beyond Compare, etc. Or even if I don't want to bother with setting up tunnels and port forwarding for testing my web app locally, I could use Firefox in the VM, etc.

Seems it would solve all the issues, which are specifically that it mostly works only with VSCode or terminal apps, but with a remoting solution you could run anything you'd want.