Sure. Specific energy (or energy per unit mass) between different types of materials makes a huge difference. For example (Source here[0]):
Material Type of generation Specific energy (MJ/Kg)
Hydrogen Fusion 639,780,320
Coal Oxidation 24.0-35.0
Note the specific energy of a Kg of burned coal compared with a Kg of fused hydrogen. Fused hydrogen generates roughly 200,000 times the energy per unit mass than burning coal.You're right. Cost and availability play into this as well. There are estimated to be ~1.06 trillion tons of coal on earth[1], hydrogen is the most abundant element in the universe and even makes up a significant amount of the mass of coal.
Burning hydrogen/hydrocarbons is, compared to fusing hydrogen, an incredibly inefficient process.
I'd say that being able to generate 200,000 times the energy per unit mass is an important consideration.
As for availability, hydrogen is more abundant and cheaper to produce (unless you have petatons of plant matter, the right conditions and a few tens of millions of years at no cost to you) than any fossil fuels. Or just about anything else.
[0] https://en.wikipedia.org/wiki/Energy_density#List_of_materia...
[1] https://www.worldcoal.org/coal-facts/what-is-coal-where-is-i...
Edit: Clarified availability.
You might try to argue high fuel energy density implies low cost, but this is clearly not the case in general.
Though it's also true that the infrastructure to process, deliver and store large amounts of fossil fuel comes at a higher cost than is usually considered (as some of it is subsidised and socialised), plus there's the pollution.
If we found a way to extract 10x as much energy from coal as we currently do, electricity from coal wouldn't become 10x cheaper, nor would we build power plants 10x as big.