Yes, exactly. I personally don't have a problem with being subordinate and receive tasks to do. But if asked that question, my answer would be just like yours. "No, I'm not coming to your place because I care about things you do (I probably won't even know what you do until day 1 of my employment anyway). I need to slave my life away somewhere to provide food and shelter for myself and people dependent on me, and your company just happened to ask me to accept you as my slave master. I have tons of my own projects and ideas that are million time more interesting than what you want me to do here, but money doesn't grow on trees.".
But of course I can't tell that to a potential boss. I need to dodge the question with "I'm a passionate programmer who loves spending time coding stuff, and I'm looking for interesting projects, which your company seems to have". Or something.
I'm increasingly thinking that maybe I should find some mundane computer work, automate the hell out of it without telling anyone, and use all the free time I gained by this to do my own projects. That way company gets the service it wants, I get paid for it and don't feel sick all the time. Win-win.
I'm passionate about a few things, but none of them include being told what to do, then having someone take a large cut out of the value I produce.
Edit: I'm currently researching alternatives to traditional corporations, like worker owned cooperatives. If you or anyone reading this have ideas or want to talk, my emails in my profile.