As for what damage you do, it kind of depends on what you're doing. But in the end you're exposing your work machine to patterns and processes outside your normal job duties, potentially exposing it and the data/access it has to additional risks.

It might be overly paranoid depending on what the circumstances are, it might be a real concern as well.