I agree with your sentiment overall, but I don't think it applies to von Neumann. The sentence right before that quote is: "With the Russians it is not a question of whether but of when." It seems generally accepted that he was in favor of destroying the Soviets before they gained the ability to retaliate, although I will admit that I can't find any more direct quotes about it.
It's hardly surprising either. During those few years when the US had the ability to devastate the USSR but not vice versa, it was a fairly common opinion. In addition to all the common reasons for advocating this, a Hungarian would also be influenced by the rather brutal domination of his homeland by the Soviets in that time.
As a mathematician, it's probably worth pointing out the use of the implication. "If you are planning on bombing them tomorrow at 5pm then why wait?" is not equivalent to "Russia should be bombed now."
And indeed, of all the logical fallacies I see on the Internet this is one of the ones that tends to bother me personally the most. "A -> B" is not! equivalent to "B". If I meant B, I would have simply said B. I suspect the same is true of von Neumann.