It's really odd and I'm better off just burning an hour or two on a more intense form of distraction and getting it out of my system. I've found it's worse when my wife is home, I guess it's an urge to keep up appearances, even if she doesn't really care? But when she's out of town I think I get more done just because I chunk time for work and play more effectively.
I've got exactly the same. I know she doesn't care, but the urge keep appearances is still there. I tend to schedule checking out videogames for those times when she takes our daughter to visit her parents - even though she would be fine with me taking a break to play games, I would feel guilty about not spending time with them instead.
Guilt is a funny thing.
HN is really the worst. With other sites I visit, I can at least honestly say it's a pure waste of time. But here, every couple weeks I'll find something that solves a tough problem I have; every year or two I find something career-altering. If I were to integrate ((value I get from HN) - (opportunity cost of time spent here)), I'm not sure whether it's even negative. My monkey brain definitely feels it's positive (magic of temporal discounting). If I spent this time on other things, maybe I'd be running a space company now. Or maybe I'd be depressed and stuck doing PHP for ridiculously low salaries.
I've traced that to feelings of guilt. I can't enjoy a videogame if I feel I'm only using it to escape and make my situation worse. I can probably enjoy a TV show a bit, for a while, until my brain realizes that it's just an escape mechanism and I get more stressed. But social media in general, offer you a bite-sized escape. There's nothing wrong in taking 2 minutes break to skim comments on a HN thread, or scroll through some funny pictures somewhere. 2 more minutes won't make a difference either. 2 more? And suddenly it's been 5 hours.
So in the end, I don't do the things I have to, and I also don't do the things I want to - all because negative emotions.
(Part of my solution to this problem was just blocking certain sites (like HN) on the router during work hours.)