That's true globally but many countries have birth rates below replacement levels. A declining population can cause problems just as an increasing one can.