Increased CO2 over extended time periods would lead to fewer large fish due to the process of ocean acidification (my understanding is that smaller fish cope better than larger fish in acid environments, that's not to say they will be prolific in an acidic ocean, however). Phytoplankton would do very well though, assuming it can adapt to the acidification and warming.

Most Earth-based counters historically have involved megaflora sequestering CO2 as they die, however this is over the scale of hundreds of thousands of years, and requires large expanses of land to host plants to do the sequestering. We've doled out most of this land already, so this isn't likely to happen. If you're looking for a deus ex machina, you might wanna hold out for a supervolcano to eject a bunch of material into the atmosphere.

Climate change models seem to indicate amplifying feedback loops would be the likely scenario rather than "countering" forces.

And for your specific example "increased CO2 -> more plankton -> more fish", this tends to not be the case in reality. Large algae blooms result in mass fish die offs. And now combine that with the the presently-occurring Increased CO2 -> warmer temperatures -> warmer water -> less dissolved O2 -> dead fish

Typically you should figure out if it's safe to drastically alter the thermal and chemical properties of the climate and biosphere before you do it. Not assume it's safe based on conjecture so you can carry on with a reckless experiment.