But yeah, on your first point there, I remember a big double-bill concert around here about ten years ago. The first half was fairly traditional jazz, well-attended and apparently well-liked. The second performer's music could best be described as a quick, semi-recognizable head followed by minutes of endless sixteenth notes without any appreciable (to me, anyway) form or structure. There was obviously an incredible amount of skill on display, but I found it unimaginably boring. I wasn't alone: after about ten minutes of the second half, there was a steady stream of people getting up and leaving during the performance. So did I eventually.
Here is "Take 5" from "Time Out" by Dave Brubeck: http://www.youtube.com/watch?v=BwNrmYRiX_o
And "Midnight Blue" by Kenny Burrell: http://www.youtube.com/watch?v=_wAmxuHt5nw
I've yet to run into anybody who turns their nose up at, say, Dave Brubeck's "Time Out", which is certainly rhythmically sophisticated but tuneful, or Kenny Burrell's "Midnight Blue" (an album that's usually the first up for visitors). It's the "I'm an artiste, dammit" attitude, the cleverness for cleverness' sake, the removal of anything remotely engaging at a visceral level -- the attempt to emulate in the jazz idiom what Schoenberg perpetrated on "serious" music -- that forces people to decide that the Emperor is without skivvies to cover his shame.
Lyrical content is a red herring as well. There's a lot of dance crap (and some that's not crap) that gets heavy rotation in the wider world that is entirely word-free.
Okay, jazz is all about painting oneself into a corner and finding imaginative ways of escaping the trap you've set for yourself, but there's no harm in playing something that manages to capture a tune along the way rather than merely tracking the letter of the changes. Too many jazzers forget that it's still music.