This is a rationalization but it's useful. We have a physical theory in L^2, and we know that you can't have such a theory in L^2.1 or L^7. Our theories might be totally wrong. But we are confident they can't be only a little wrong.
The concern of "is this thing square-integrable" arises only in infinite dimensional spaces. Inner product of all $\mathbb C^N$ where N in finite is finite. There is no point in saying "L^2" when talking about spins etc. (forcefully casting a finite sum of finite terms to integration and then introducing the concept of "square integrable" isn't helpful, if not trivial).
> This is a rationalization but it's useful.
You can do that kind of rationalization when you "know" the answer ahead. It has zero prediction power.
Newton had a rationalization about the independent nature of time. Descartes also had a rationalization about how action-at-a-distance works for gravity. Aether was also a popular rationalization once. None of those rationalizations held any value at the end.
Such "rationalizations" are better cut by Occam's razor and left for philosophers.
At least, I prefer not waste my time with such stuff as a physicist.
Your mention of Occam's razor is key. The point of this rationalization is that it shows quantum mechanics (or at least a subset of the theory) is mathematically simple, thus serving a threefold purpose:
1) Due to Occam's razor, a simpler theory is more likely to be correct than a more complicated theory with the same explanatory power. If quantum mechanics can be expressed more simply, then the estimated probability of its correctness should be increased (by only a slight amount).
2) Quantum mechanics, explained this way, is interesting from a purely mathematical standpoint. Even if we knew quantum mechanics doesn't describe reality, mathematicians (and theoretical computer scientists like Aaronson) would perhaps still investigate it. Of course, this may not interest physicists.
3) The article presents a novel way to teach quantum mechanics. Mathematical simplicity can (to some extent) replace intuition as a way for learners to grasp the theory. As Aaronson remarks, quantum mechanics is often taught by following the historical order in which the ideas discovered. Starting from the "conceptual core" (if Aaronson is correct about the conceptual core) is arguably a superior pedagogical technique.
"Simpler" theories are not ab initio more likely correct. The world is full of "simpler" theories that are wrong: the four elements, the caloric theory of heat, Newtonian dynamics and gravity, and so on.
"Simpler" is a purely human notion, and a heavily culturally laden one at that. For this reason, Occam's razor is best left in the dust-heap of philosophy. It is never useful in doing actual science, except now and then by accident.
As for the notion of simplicity itself, we have Kolmogorov complexity and Solomonoff induction, they capture the essence pretty well (though I heard that with some caveats).
The problem with all your examples is that Occams razor says "choose the simplest theory that works", not the simplest of all possible theories.
I explored this idea in more detail here: http://www.chrisstucchio.com/blog/2014/why_to_reject_complex...
Don't take his word on how quantum mechanics is thought. While we don't teach undergrads it, there is a conceptually simpler approach to quantum mechanics: Feynman's space-time approach. Not only it is conceptually simple and intuitive, it most importantly gives an elementary understanding to the principle of minimal action (sum over all possible paths in space-time) but it also offers a way of calculation that is much better suited to certain class of problems. Now, that is useful.
If you're a layman, you can read "QED: The Strange Theory of Light and Matter". If you know some physics and maths, you can read: http://www.feynmanlectures.caltech.edu/III_03.html
That is how quantum mechanics is made simple and intuitive (and that is superior pedagogically). Not by introducing new and strange additional concepts such as negative probabilities as basic things just because you can.
The reason we don't teach path integrals in the undergrad is, we expect students to actually use it for calculations, and unfortunately the mathematics is much more involved in comparison to matrix mechanics or wave equations where you can get away with "basic" maths. If you have a very good intuition and understanding of quantum mechanics, it is doable though, see Feynman Lectures on Physics Vol. III (not sure how many percentage of the students will actually be able to absorb the intuition along with the new information though).
But I don't think that's even the point. The whole article is a rationalization of basic rules of quantum mechanics; when I look back at the history of physics, rationalization of known --at the time-- physical laws (which are often replaced by "better laws" after a while when get a better we understanding of nature) is most often counter-productive.