Er, yeah, of course you're right.
Adding that factor in will mean adding in even more fake, hand-wavy numbers, but doesn't really change the end result; whatever value you end up with, if you double the weight of the batteries, you will need to displace twice as much air (or move it twice as fast) to compensate, which means you'll need to use up twice as much energy, leaving you with the same flight time in the end.
> let's just round that to 10 m/s^2. 10 m/s^2 100g is 1 N, which means it requires 1 W of power to keep that battery aloft*
To transform some force into power, you need to multiply it by a speed (m/s). My favorite example is that a brick resting on a table can "hover" without spending any power (for a weird definition of "hovering").
I think the correct calculation involves the density of the air, to estimate the speed of the air that must be pushed down.