I assume it's because he/she is spending time doing the side projects, time which could instead be spent doing whatever he does for his job.

I feel like this is actually one of the things I really hate about being paid salary. I either feel guilty for not working enough or just hate not being able to spend time doing other things.