Unless we reduce our use of electricity or come up with a new source, those panels will always be there.
It will also take time for the wild life to grow back and if the the ecosystem has changed enough it won't come back the same. This isn't a problem by itself. But if Chernobyl is an example all you have to do is wait 50 years and wild life will move back in too.
I'm 100% for renewable energy, but we're making a lot of assumptions on how we're going to get rid of and recycle solar panels. That's a lot of material we have to re-process and see how much we can use and how much was consumed.
The reason these plants are reasonably cheap is that there are rows and rows of uniformly hooked up and distributed easy to access solar panels.
What you're suggesting is best accomplished by a government subsidy on solar panels. But then individuals own them and we won't have a way to recycle them when they all go bad in 30 years.