”Punctuation is extremely important to me, but I insist on doing it my own way.” — Knuth
[Edit: BTW, where in the paper are you seeing "plenty of indications to the contrary"? As far as I can tell there is only one earlier program by von Neumann mentioned, an earlier and 80x slower version of this same sorting program, which has not survived.]
I have received one actual email from him, when I pointed out an error in the balances at the Bank of San Serriffe (https://www-cs-faculty.stanford.edu/~knuth/boss.html).
The email looks like it was typed by him - the style is right and it is signed "Don" - but the From: address has someone else's name.
"Stored program computer" refers to a computer where the program would be stored within the persistent memory of the computer itself, rather than elsewhere. It seems hard today to imagine such a thing, but think about punch cards or programs you have to enter in via adjusting switches or plugboards. The point of a stored program computer is that program running on the computer could, in theory, change the program itself (yes, "code is data" predates lisp, sorry).
According to Jean Bartik[0], Mauchly and Eckert's EDVAC was already a stored program computer, and thus implying that one of them must have written the first stored program. Jeremy Norman has the actual document where this happens[1], months before Von Neumann's publication. There's some evidence that some of the ideas in Eckert's paper were from an even earlier paper from John Atanasoff. In any case Bartik has a very poor opinion of Von Neumann, who apparently sat in on EDVAC meetings and wrote notes on the EDVAC without crediting Mauchly and Eckert, then allowed those to be distributed widely in his name, which in turn invalidated Eckert And Mauchly's patent application and thus also their claim to the invention in the public eye.
Another interview with Eckert[2] gets into the details of these allegations: That Von Neumann's motivations were that things should became public domain was because he specifically couldn't sell those ideas at IBM (for whom he was consulting) unless they were patent free.
Curiously that interview also has a reference to another person claiming that a lot of papers attributed solely to Von Neumann were actually coauthored. Really gross stuff.
All this reinforces my notion to be more suspicious of modern day "first" inventions attributed to a single person. Similarly to the telephone and lightbulb, there seems to have been a very active community of inventors (all quite important and impressive) who were dancing around related ideas and another might have gotten there eventually if one didn't. There are about 7-8 early computers, some of them within months of each other, that could be considered early stored program computers. I'm not sure it makes sense to worry about which one was truly first.
[0] https://news.ycombinator.com/item?id=40333159 [1] https://www.historyofinformation.com/detail.php?id=639 [2] https://conservancy.umn.edu/server/api/core/bitstreams/7de38...
Maybe Knuth will issue a reward check to someone who emails him to fix the title?