Hmm... or maybe some smaller panels plus a really enormous and very very thin reflective collector for concentrating solar. Keep in mind that things can be very thin in space.

It would still weigh a lot though, and heat would be a major issue. You wouldn't want to melt the solar panels.

This would require someone much more knowledgable to determine if it's feasible.

Edit: I wonder if you'd get some amount of solar sail effect in addition to the plasma rocket? Of course this would help you one way and hurt you the other way.