/usr/bin/git is usually a soft symlink, so the original binary is still there when you rm the synlink. You can see the actual location by doing

    ls -al /usr/bin/git

If it's not in fact a symlink then yes you should probably rename it to preserve it.