If you're talking about traditional UX values like intuitiveness, sure, vim UX is terrible.

But vim features tend to do very simple things, which are composable into very powerful mini programs in a lot of contexts. There's a sort of "discoverability" to them as well, in that if a key does something in one mode, it likely does something intuitively similar in another mode. All this adds up over time. I've got about 9 years using vim every day now, and the combined learning over that time has added up in a way that disposable learning like "how to use the X flavor-of-the-week plugin" would not have.

Another thing here is that the UX improvement of most plugins is only surface-level. vim features tend to be very simple, which means that when you understand it, you understand it. But vim users with a lot of plugins rarely understand what is actually going on when they use a plugin feature, because the features do too much. This means that they can't really use those features in, for example, macros, which loses you a lot of the editor's power.