The CO2 is released when (and if) the tree rots.

Dead plant matter used to become coal. Dead plant matter today decomposes:

"Based on a genetic analysis of mushroom fungi, it was proposed that large quantities of wood were buried during [the Carboniferous] period because animals and decomposing bacteria had not yet evolved enzymes that could effectively digest the resistant phenolic lignin polymers and waxy suberin polymers. They suggest that fungi that could break those substances down effectively only became dominant towards the end of the period, making subsequent coal formation much rarer."

https://en.m.wikipedia.org/wiki/Carboniferous See Rocks and Coal

That depends on how the tree decays. There is a lot of coal near the surface of the western US because prairie grass burned yearly in every dry month (winter when it was too cold to grow - they may still be as much rain/snow!). This yearly burning didn't get warm enough to burn all the carbon and so the rest got deposited as charcoal.

Moral of the story: you should start regular forest/grass fires to turn the stored carbon into charcoal. A biologist can give you a number of other reasons why this is good for the forest as well: common knowledge about forest fires is almost completely false.

Imagining a forest where dead trees never decompose is quite a mindbender. Certainly a better explanation for the presence of fossil fuels though than just a handwavey mention of geological timeframes.

Sure, but before then it's stored. And if trees are constantly creating new biomass, you've perpetually buffered an amount of carbon equal to (annual production x time variable). The first variable is limited by the thermodynamics of sunlight, so do what you can to approach that limit. Then the main target of innovation becomes, what forest management techniques can maximize that time variable?

By default about half of a tree's absorbed carbon is injected into the ground to build soil, as root mass and root exudates. Plus trees drop mulch -- mainly leaves. Soil carbon is only released when the soil is destroyed or eroded away. So... don't do that. :)

Some of it is retained in the decayed remains (eventually soil) but much is released in the decay process.

It's worth noting that the decay process takes about as long as the growth process for some trees. For example, a Douglas Fir might spend 600 years alive, and then be a Nurse Log for another 600 --- see http://www2.kuow.org/program.php?id=26100