But they don't.
There are media for longer form presentations of ideas -- dissertations, books, and lecture notes (a.k.a. early drafts of books) come to mind. Each of these is de facto mandatory for a successful academic career in science. A lot of academics also publish blogs, documented software packages, etc.
The complaints about jargon in journal articles and conference proceedings from non-scientists or from scientists talking about an area other than their own come across, to me, as kind of self-centered. Sometimes laypeople (or even scientifically literate generalists) aren't the target audience of a text. That doesn't mean there's some big problem with the world.
If I had a dime for every time someone who's never taken a non-intro science course complained about not understanding a scientific publication...
You talk about making no assumptions. I agree that this is excessive, but why can't an MSc within the relevant area be enough? Yes, you save some space by using a lot of jargon, but if you use a word only once in your article, would it kill to spell it out instead? And while jargon occasionally saves space, snobbish language never does.
I think the points about specificity and word count really explains this. Snobbish language is going to kill word count except if you're talking about words used that are not required for specificity.
> Bosley (...) says that academic prose is often so riddled with professional jargon and needlessly complex syntax that even someone with a Ph.D. can’t understand a fellow Ph.D.’s work unless he or she comes from the very same discipline.
That doesn't necessarily mean that the article is "needlessly complex", it could also mean that the article deals with a complex topic, and therefore will be complex to read if you are not up to date with the topic. An article that makes no previous assumptions is ultimately called a book, and runs hundreds of pages long, as opposed to an 8-pages scientific article. Or, to make an analogy, it's not intended to be the full source code but a diff to previous research.
Of course, there is some bad writing around (like the IG Nobel prize, where students use longer words just because they can). But to claim that it makes for a majority of writing seems to me like a stretch.