All that the "Big Bang" refer to is 1) the fact that the (visible) universe was once much hotter and denser than it is now and 2) if we extrapolate back in time, using the theories that we have, there was a point where the density was infinite.

Of course, we know that our theoretical model break down before that (since we don't have an experimetally verified theory of gravity compatible with quantum mechanics). Physicist that work in the field (and I used to be one of them) are well aware that we cannot extrapolate past the regions of validity of our theories and no one claim that there was indeed a singular point ("Big Bang") as mentioned in descriptions aimed at a non-expert audience.