The main process for polysilicon manufacturing (Siemens process) uses electricity. That doesn't need fossil fuels.
The step before to reduce the silicon dioxide requires carbon, which is where the fossil fuels come in.
Same thing with steel – both are critical input materials and can be made without fossil fuels, but they aren’t today. Maybe a carbon tax would fix that!
Are solar panels convenient? All polysilicon today is made with fossil fuels, and the R&D to make it with renewable energy is still in-progress. Not to mention that we ship them across the ocean with fossil fuel.