The manufacture is indeed the usual silicon process which involves toxic chemicals, but they don't have to end up in the environment.

The cells themselves are pretty inert, although CdTe (rarely used but once popular) is quite toxic. And of course none of this stuff radiates.