It's limited by the fact that you need a very large volume of lighter-than-air gas to lift a human. The theoretical maximum lift capability of 1 m^3 of hydrogen is just about 1.1 kg so you will need a really big airship to carry a passenger.