Personally, I don't find watching sports to be very engaging either, but I know I'm not typical in that regard. Theoretically, I can see more of an argument in favor of watching traditional sports since it's harder to go out and find other players of similar skill in your location and then organize everyone into teams, etc, etc. The barrier to participation in video games is much lower though with the internet and automated matchmaking for multiplayer games. Singleplayer and free-to-play games are even easier to get into.

I love playing video games, but I've never had the patience to seek out and watch other people play. I'd much rather be experiencing it for myself - but as you said, to each their own.

Another analogue would be watching people play chess. Many people find it interesting to watch people who are good at what they do.