To illustrate you how badly skewed it is: for a few years, me and my partner had a foster home for stray kittens. In Poland. That means lots of hard work that rarely pays off (by having a healthy kitten that gets adopted, there is no money in it, obviously), depressing amount of death and suffering, participating in such great activities as autopsies and interventions.
It was a significant part of our lives, our passion and when it paid off, it brought immense satisfaction. It was a very natural subject of talk for us, but really -- do you want to hear about the state of an inflamed heart we found in a dissected kitten, what happens to bodily functions when some dude flails a cat while holding it by the tail, how a neglected cat lost her eye to disease or other fascinating subjects from this category? All those conversations while eating?
Then why should people who aren't passionate about children endure conversations about children defecating? Really. I can handle it, of course (while I don't love it), but I can repay with a story about a dog eating her own intestines, so I'm probably not a benchmark for most sensitive listener.
People not wanting to hear about babies are just like people not wanting to hear about eyes falling out of sockets, it's only ours (yeah, I can talk about babies too) presuppositions that make the subjects seem so different.
(btw, if you rob me of my daily walk, I will fall asleep in your office)
Nothing wrong with going for a walk, either. I would usually eat lunch alone when I was in a corporate environment. It's really just the anti-social and frankly mean approach taken to normal conversation and interaction by edw that put me off.