DE being the first thing a common user interfaces with, it holds a very important position, in my opinion. KDE and Gnome are very different w.r.t design principles, hence the polarization and debate. KDE and Xfce are not so different. I like Gnome, but have no use for it.
I agree with all else.
Also, I think, a real "free market" would resemble Linux ecosystem. No monopoly, small players and lots of choices. However, if we wish to compete in "real world", we need to do something similar to what you say.
Consider the possibility that your idea of what "the common user" wants is something you made up.
I would argue that having a choice between hundreds of DE's is a bad thing for the more casual user.
The large majority of their users don't even know what kernel is running the runtimes of their apps.
One of the reasons of netbooks failure, besides the cheap Windows XP licenses move from MS, was that each netbook Linux was its own little silo.
Every distribution spends more time on packaging and integration testing than developing useful functionality. When developing for Linux you need to package your application in a few different formats. We also have different init systems to make it more difficult. Every distro has own store and repositories. For GUI there are different frameworks and display servers.
If Linux world standardizes around snap/flatpak, systemd and wayland then we stand a chance.