Climbing gyms? They're plentiful these days, so chances are there's at least one in your town (or seven if you're in a major city). $15 will get you a daypass to try it out, and it's comparable to any other gym membership in cost long term.

As to climbing outside, rock shoes cost $150 and the internet will tell you everything you need about where to go and how to do it.