My personal philosophy is all things in moderation. So if it makes him happy to watch the Wiggles or some old-school Fraggle Rock on DVD, he gets to. But it's the responsiblity of me and mom to keep it reasonable and keep the house a happy place for everyone.
For example, suppose a child wants to read a good book, you take it away, and he "gets over it in seconds". Does that mean all is right with the world? No. Whether or not he remains upset about the book, he's still missing out on a wonderful book that could have benefitted him.
One also ought to think about how to correct errors. For example, pulling the plug on the TV avoids any input from someone who might know something you don't (like, your child might know more about how important it is to him to see a particular show). Wouldn't it be better to reach an agreement? The more right you are, the easier that will be, won't it? But if you have made a mistake one time, then getting others to agree will be harder, so you get an opportunity to avoid that error.
Just do it.
After Katrina, when we evacuated to my parents' house, they'd just moved in themselves, the day before. We agreed to not hook up cable. My kids calmed down noticeably. No, remarkably. We watch, maybe, a movie or two a week. The DVD player broke, so they only watch those movies on a computer anyway. The TV hasn't been turned on in weeks. I now own a $5000 Ethan Allen box to hide a $500 TV box that I don't use. Just throw the boob tube away. What do you do with a livingroom that has no TV? Feng shui.
My wife has started advising parents of kids with ADHD to get rid of their TVs. They're scared at first, but they do it. And in a few days or weeks they love her for it.
The pediatric neurologists I was with last month think ADHD probably isn't even a legit diagnosis (psychiatric diagnoses are voted on). They very frankly told me they think it's mainly a combination of TV and bad parenting, which is on the rise thanks to TV and entirely too many double-income homes. And if you're questioning their politics, these folks are insanely smart and educated, and probably more liberal than Noam Chomsky, and I know at least one that raised four kids herself.
In case you missed it: throw away the TV. All of them.
For specific readings, go to PubMed's Entrez site and look up the author "Christakis DA". He's prolific, but his biggest newsmakers are some rather large studies showing dose-dependent correlations between TV and obesity, attention deficit, violence at school, and I think there was something else.