Many of the casters are skilled players themselves. Artosis (one of the commentators for GSL) was quite highly ranked. I believe Tasteless and his brother Day9 are as well. What's interesting to me is Husky, who I don't think is anywhere near their skill has the largest audience.

I've got friends who don't play starcraft who watch husky and love it like it's any other sport. So I definitely agree there is a lot of skills involved and even different target markets. The hardcore players like the best player/casters. There appears to be an even larger casual/not playing at all audience that enjoys the sheer entertainment value of esports.