Op's point is that the lower nominal voltage is the culprit in the battery's lower energy density. If you and I each have a 1 kg battery that stores 1 amp hour, but yours puts out 3.7v and mine puts out 2.5, then your battery is storing more total energy than mine. We can both manipulate that output voltage (via boost converter, or more commonly just using multiple batteries in serial) to get whatever we need, but given the same load, my battery's just going to drain faster than yours.

This, plus also boost converters are only 85-95% efficient and they are very expensive.