> One of your examples, swimming, will put on weight as it will builds muscle mass which obviously is heavier

Sure, but what you want is more muscle mass and less visceral fat so that's exactly what your want. Raw mass is too coarse grained to tell you much.