The confusion likely stems from Ammonium Nitrate (uncontaminated) generally only being explosive when hit with a very high speed shockwave (aka a booster charge + detonator in typical mining/blasting applications), or when sensitized with other chemicals (like with tannerite).

Most of those other blasts, including Beirut, involved lots of contamination of the AN + large fires + very large quantities of ammonium nitrate.

In truckload or less quantities, and when not contaminated with specific substances, ammonium nitrate is pretty much impossible to detonate without a sizable quantity of some other high explosive.