more details here:

https://en.wikipedia.org/wiki/Flame_arrester

https://en.wikipedia.org/wiki/Davy_lamp

https://www.youtube.com/watch?v=u-GeraWUP6E

The wire mesh drops the temperature below ignition temperature and also drops the pressure. Since the outer air remains relatively cool, the mesh presumably continues to shed some heat through contact with the outside air, and so doesn't itself reach the ignition temperature on the outside. Maybe sort of like how you can hold a flame to a paper cup full of water and even though the paper is dry on the outside, it won't catch.

I suspect that it acts as a sort of diffuser. Any source of ignition within the grill has to pass through it to induce an explosion. The mesh allows the heat to spread out more evenly, making the temperature at any one point on the mesh lower (the energy is still there, but likely isn't concentrated enough to cause ignition). As a result, with respect to the flame, the little burning particles from the wick need to get through the grill to ignite the gasses outside the cage, and the fine mesh makes that less likely too (I think microscopic burning embers are surprisingly likely to bump into the mesh, since fine wires have high drag aerodynamics and so the air will cause the gases to linger near the mesh; a finer, denser wire mesh just amplifies the effect that way while still letting out much of the light).

I bet it's not failsafe, but it undoubtedly is a vast improvement on an open flame.