You've got a bunch of poor people starving and set up a factory to produce a couple of bombs and pull some farmers off the land to have them march up and down the parade grounds. That isn't going to put more food on the table. Sure it might be a good idea regardless because being invaded sucks, but it isn't wealth creation without a very creative understanding of wealth. A society can't be better off without creating new wealth.
I suppose there could be some sort of plowshares-to-swords program where they build tractors and bulldozers that can be repurposed into military gear on short notice. But I havn't heard of that working anywhere before and my read of modern militarys is that the gear gets pretty specialised to military use. And the soldiers need to actually focus on soldiering rather than being part timers.
At some level a country needs an army because without one they don't get to have an economy, but it is a dead weight in terms of prosperity. Military expenditure comes directly out of what people would otherwise use to improve their own lifestyles.