Higher capacity you mean (for a given volume), that and also very low leakage current so if you want to make a very efficient device then tantalum might pay off. Another reason is that electrolytic caps are usually wound aluminum sheets which act as a coil. If you don't want any parasitic inductance tantalum is better in that application (better frequency response).

Tantalum caps have much worse leakage current than ceramics.

The debate for most designs is not tantalum vs. electrolytic, it's electrolytic vs. solid vs. ceramic. And ceramic usually wins.

They have lower internal inductance and resistance than other electrolytics. That is very goo for making the low-pass filters one uses on power supplies.