The problem is we are literally digging carbon out of the ground and putting it into the atmosphere, thereby increasing the amount of atmospheric carbon over time.
As but one factor, dying plants would release CO2. The total amount of carbon in the atmosphere is (was) at equilibrium.
For a somewhat similar scenario, you can read about the Carboniferous period https://en.wikipedia.org/wiki/Carboniferous . Essentially, the plants invented the wood but nobody know how to eat it, so the dead plants accumulated sequestering a lot of CO2. So the CO2 concentration dropped and the O2 concentration skyrocket. As a side effect, the high O2 concentration allowed bugs to became giant. (It ended when fungus learned how to eat wood.)
[1] And all the other animal, fungus and even plants. And also bacteria and archaea.