Luckily the war didnt happen, but it was a reasonable possibility. So only in hindsight do these beliefs look stupid (and evil).
Von Neumann also influenced the Goverment on Military Strategy and basically invented MAD. Which worked great so far.
Considering the oppression that his native country experienced at the hands of the USSR, and the fact that the implications of nuking an entire country weren't really understood yet at that point, this view is hardly surprising.
Consider that Bertrand Russell - the well-known and self-described pacifist - shared that view as well.
To greatly simplify the concept: if you believed that total war between the USSR and the West was inevitable, attacking first while the US was the only nuclear power would substantially limit casualties and ensure that the West won.