You're not wrong, but this is also an argument that you shouldn't ever be allowed to do anything dangerous or risky.

We allow all sorts of dangerous activities with the same problems. If we're worried about the rescue and medical costs, we should definitely ban skiing, skydiving, climbing, etc.