... But if "cost wasn't a factor" then we could just simply dedicate the electrical output of fusion plants to brute-force CO2 out of the air.

But there are other environmental impacts. Fusion, even compared to fission, has an extremely small footprint (per megawatt). Its fuel is easily available (isotopes of hydrogen). It does require some pretty advanced magnets though, so it will contribute to the strip mining that we do for rare earth materials (though this applies to all energy forms, including solar and wind). I don't have numbers to say if a fusion reactor would use less total rare earth metals per megawatt compared to something like solar or wind.

One thing to note though. Once we get sustainable fusion reactors, it will still take a bit for that cost to come down significantly. That usually takes 10-20 years. This is a pretty common pattern. We've seen it from the price of laptops and cellphones to the price of solar panels.