I understand what you mean. While I'm a Factorio fan I felt this way about the graphical-programming zachlike SpaceChem. Too much like work.
I approached Factorio like MineCraft but with the boring stuff automated.
To have fun in a game like Factorio, just mess around - you don't need to make a perfect factory, or compete with your friend.
One reason why I can't really get into Factorio. It's too much like work.
The funny thing is I do code at home, for fun, in my free time- or I used to when I had a clear line demarcating my "free time" from work time (I do AI research and I mostly work from home, so most of the time I'm working on something that I really have to work on). But Factorio is just ... too much like work.
... dah, I can't explain it :)