At the expense of maneuverability. It's worth mentioning that this is the trade off here.

From least to most maneuverable:

Weather balloon -> Fixed Wing -> Single rotor -> Multirotor

Definitely. Now I want to build a blimp drone or a blone if you will just to give it a try :). I imagine a toroidal shaped ballon with a small propeller in the middle might actually do fairly well and would be less susceptible to sideways wind than a balloon.

This was posted in the discussion when the Lily was announced:

http://hyperblimp.com/

Yes, definitely, but scaling volume to carry payload mass becomes inconvenient for casual use quickly.

Cheating the calcs a bit and using the table here: http://www.chem.hawaii.edu/uham/lift.html A 3.5oz (~99gram) Go pro needs at least a 22in diameter spherical ballon filled with Helium. And unless you're ok with tethered use, you need to dedicate some power & mass to pointing & station keeping which increases the volume even more. For every gram of other capability you add, you're adding another liter of gas.

So you need roughly 100 grams of payload + probably another 100 or so grams of mounting gear (gimbals, etc.) + directional propellers + battery packs which likely total out at around a kilogram. So that's roughly a 1.2 meter diameter balloon or 950 liters. That's fairly large, I agree.

Do you know if a hybrid has been done where you have a multi-rotor assisted by TLA ballast?

None that I know of, but multi-rotors lateral flight authority is often weak vs winds. So, I would worry that a big gas bladder might strongly exacerbate that by requiring a much higher capability vs wind. But then again, a hybrid might not be a 'conventional' multi-rotor config.