Sure, everyone loves if stuff just works - but if I'm looking back to my first years of Linux (1998-2007) and the last (nearly) decade - I haven't actively cared what kind of kernel, what version of kernel or whatever I am running on my laptops or servers. It just works[tm]. I'm really not trying to downplay the kernel developers' work - but if you're not using the latest hardware or need the last bit of performance.. many people could live with security fixes alone (hey, surprise, all the kernels in LTS distros only do get security backports).
(For me at least) in the last 5+ years the real development was in userland (for better or worse) - but I've stopped caring for anything besides "works fine" on a kernel level. Sad? Maybe.
The desktop devs on the other hand seems all too willing to break existing interfaces, treating them more like internal code than something exposed to third parties.
I think a big part of the problem is Red Hat. Not that they're malicious or doing something wrong or anything of the sort; just that they're overrepresented in key projects that determines the direction of Linux. With enough people in key positions it is much easier for others to write off working upstream because they have different business interests and feel like contributing is working for you rather than with you. Here LWN with their stats on contributor affiliation for kernel releases can help (since they show that they're not even mostly one company), as does having people employed by the Linux Foundation. This doesn't occur as much for non-kernel software though.