There have to be rules where those taxis can start, land, in what direction, time frames and over which places in a city. You need some kind of air control and "traffic rules". Drivers need regulated training in simulators and there need to be emergency procedures.
I believe that in the first few years such things won't be allowed near towns and cities. There will be similar rules like flying drones [1], but a lot stricter.
And the time saved by such travel will probably decrease when the sky starts looking like in the 5th Element [2].
[1] https://uavcoach.com/drone-laws-in-switzerland/ [2] https://media.giphy.com/media/Bs2pZhpxf2168/giphy.gif
Short term, these things will mean helicopters that currently service rooftops in many cities will be replaced by slightly more of these things flown by professional pilots that will have increasingly less to do as these things start flying themselves. I also expect an increase in heliports. E.g. Manhattan only has a few right now.
By "energy cost" do you mean something like "the price of a cup of coffee spent on electricity"? That seems surprisingly efficient but maybe here (Europe) the coffee is cheaper and the electricity dearer - what's the figure in kWh?
By the time these things start flying at scale, the coffee will be way more expensive than the energy cost. That's why most of these companies are investing in autonomous flight as well because short term that is going to be the cost bottleneck (that and training the army of pilots that they'd need).
The reason this will take off (physically and economically) is cost. If you can travel 40-50m quickly for about the energy cost of a cup of coffee, it's going to be a game changer in big cities. I can take an Uber across town in Berlin but it will cost me around 20-30 euros and take 50 minutes (worse in rush hour) and is not that competitive with public transport. The same journey with a flying taxi could be done in 5-10 minutes (just like with a helicopter) the difference will be vastly lower cost. The main cost will be the pilot who can now do multiple journeys per hour and at least short term still charge a premium.