I don't understand your complaint. Papers are written by researchers to each other. The 'correctness' and 'proofs' are the point of the paper. The standard is the simplest possible explanation that includes the justification for all claims made. You are asking researchers to write another, different, article for you. Maybe they would be better spending their time on more research and leave the popular writing to others who might be better at it.
If you are asking for some explanation that lies in between the extremes of original paper and a popular article, you are in luck: this is what textbooks do.
The supersmart ideas usually spread out before the textbook gets written - grad students bring them along as they go into industry, etc. Most - almost all in fact - papers are pretty boring by themselves. The ordinary case is that papers gradually build and refine ideas for a few years until we look back and say 'wow, we made some improvements on a decade ago. Cool. Now, pushing on....'
This idea:
"revolutionary papers present ideas simply and are overlooked, often because of the misguided notion that 'simple to understand' equates to a 'trivial insight'"
is popular in the imagination but I don't think it happens very often. More common is when a great new idea is expressed wonderfully simply and people skilled in the art read it and say 'holy shit that's neat'. For example Einstein in 1905 was a near-nobody that presented powerful ideas very simply and his papers were not passed over.
2. A ton of research papers are extremely dense, while the actual 'newness' could be explained with a simpler definition and a really good diagram or code.
3. Without making too blanket of a statement, often, formal proofs are much more useful after one understands the intuition.
Inotherwords, I'd like to see a format that stresses expressing the 'new idea' simply. Proving that the new idea is supported by mathematics is definitely important (as you point out - its the 'meat'), but I believe understanding it follows intuition in many cases.
I agree with your other points, and many authors do work hard to give an intuitive description. Intuitive, that is, to a reader skilled in the art.
But don't think that just because you don't understand a paper that means it is overly complicated or obtuse. It might be perfectly compact and straightforward if you are familiar with the field and its conventions and notation. And this is the most efficient way to communicate.
Of course, some researchers just suck at writing. A few might even be trying to sound fancy and make things sound complicated or high-falutin'. Grad students start out with this tendency, but we try to beat it out of them ASAP. Some are never redeemed. But I like to think my group produces readable papers.
No, it is all that matters. Having a neat and intuitive understanding of a new idea that is wrong has at best no value and at worst is actively harmful.
Isn't the audience of an academic paper other academics? Placing proofs and details in a separate section is convenient for the layman but bothersome for the intended audience. Often, it is precisely these details that are important. In mathematics for example, using new methods to give simpler proofs of old theorems is very useful; here, other researchers would care about how particular details are resolved.
I could draw you a picture of my latest papers if you wanted, and it may look pretty but I think you'd be cheating yourself if you thought you were getting anything of out it.
If I can't get the key value of the paper from that first readthrough the paper is usually not very good. Writing a good paper is actully hard and the typical formats exist for a reason. If you review say 20 papers/day during your initial research phase it's more valuable to have clear structure and an abstract than to have "easy to understand" language with lofty examples.
So basically...good papers do provide the simplest explanation possible. In fact it's something you very actively try to do when writing a paper. Or in other words: I think you just want more papers to be good (there's a lot of unreadable crap that seems pseudosmart but ask most academics and they'll tell you they strive for easy to understand).
I do think there's a problem with how research gets shared more generally though. I wish there were more scientifically literate writers at that boundary, because unfortunately, there aren't enough hours in the day for researchers themselves to fill that role.
Academics want the credit for their work, which I think they should retain. However, dissemination of work is lacking - there are tons of supersmart ideas locked in academic papers; they act as a barrier for many.
Instead of a dense academic paper in LaTEX or whichever, what if the standard were to provide the simplest explanation possible, that required a graphic that demonstrated the idea.
Its sort of a tragedy though, when revolutionary papers present ideas simply and are overlooked, often because of the misguided notion that 'simple to understand' equates to a 'trivial insight' - intoerhwords, if you read it and it makes sense immediately, sometimes you think that it musn't be revolutionary. This doesn't always happen, but definitely to an extent.
All of the 'correctness' and 'proofs' are useful for a separate crowd, and should also be included, but in a separate section because they are for a different user, namely, for other academics who are well versed in the domain. Also, a place for code/data/materials, as well as a checklist or script or otherwise to strictly reproduce results.