There's nothing wrong with skateboarding, or with a lot of other "layabout" sort of activities. The problem is that people aren't being given public spaces for social activities, leaving them with the choice to do them in a space that's annoying for others or not at all.

Skateboarding, in fact, is one of the least impactful of these activities, which is part of why it's relatively popular - you can actually do it within the confines of the modern cramped city, as opposed to, say, baseball, or frisbee. What do you think would happen if a bunch of kids started playing hockey on a city street?