If it's heavier it's actually more stable because it's harder to move a heavier object. But maybe you mean that a heavier craft is slower to stabilise?

I think it's because the batteries are teetering above the centre of gravity?

you want center of gravity above the propeller, its the classic balancing a broom on the tip of your fingertip trick