Except, of course, "salle de sport" is not an activity, and if you want to be perfectly correct about activities you should say "s'inscrire à la natation" not "à la piscine". "Piscine" (swimming pool) is the place, "natation" (swimming) is the activity.

It's fine to say "s'inscrire à la salle de sport/piscine" if you are talking about a specific one.