Of course when you start out reading notes, double-sharps are confusing as heck.
I do still think that in the end people are just conforming to the standard to make life much much more easier, and that it isn't so much that certain things are better for the expert then the novice, but that experts are more used to the standards than a novice is.
I feel it's more of a matter of personal style than a programming language x being more efficient when you become an expert when compared to language y. Some people really fly with Perl, whereas other people are more of a Python kind of person.
About the double sharp, it often has to do with tonality. Classical music is typically more tonal than jazz. The sharp is needed when you temporarily change tonality, and some tonalities just need double sharps then and there –Gsharp dur for instance. If you wrote the natural note instead, it would not convey the intended tonality.
I was also unnecessarily harsh towards double-sharps as well, when I was learning piano I would sight read a lot of easier pop/rock sheet music, and there were a few pieces I encountered with an unnecessary double-sharp that would catch me off guard because it was uncommon for the style.
I don't feel that any written music can or could adequately convey the feel, shape, or nature of the music--you've got to work through it with the group, or hear someone who already understands the piece.
Most jazz musicians I have talked to and taken clinics from really don't like sheet music though, as it is an ineffective and inconcise way to represent a tune's structure and melody, hence why they prefer charts. Charts really are worlds ahead of sheet music in my opinion.