While it's true that most explosives use nitrogen-based compounds, only a vanishingly small amount of all nitrogen based compounds are used for military applications. Rough estimates are that about 85% goes to producing fertilizer, most of the rest is for various industrial applications and as a fuel.

Relative to the scale of how much fertilizer humanity uses, there's just not all that much demand for explosives.