I continued to do physical activity. I think your muscles will "shrivel" if you don't use them; they won't be engorged with blood on a regular basis, thus will lose plasticity at a cellular level. Then when you do use them, they'll fill with less blood, and look smaller. It takes a lot to actually lose muscular tissue. Mostly people just lose muscular plasticity, and they appear smaller.