Rocket engines are actually extraordinarily efficient. They're the most efficient heat engines we have. More than 90% of the thermal energy produced in the thrust chamber is converted to kinetic energy of the exhaust jet (especially in high expansion ratio engines). The fraction of chemical energy of the propellants of a launch vehicle that ends up in the kinetic energy of the final stage is surprisingly high (well, it was to me, initially.)

They are not efficient at doing their job (moving things upwards), hence we don't use them to get from the bottom of a skyscraper to the top. They have to carry their entire mass of fuel with them.

A large crane is far more efficient at lifting items.

But even if you accept you have to use a rocket, the cost of the fuel is trivial compared with the cost of launch.

Of course they are not efficient for that use case. That use case is not the same as the use case of getting to orbit. When the delta-V involved is similar to the exhaust velocity of the rocket the overall efficiency goes way up.

If you could vary the exhaust velocity of a rocket continuously, so it was equal to the total delta-V so far, the efficiency of converting jet kinetic energy to vehicle energy would be 100%, as the jet would be left stationary in the reference frame of the launcher. (This ignores gravitational potential energy and also that the mass ratio would diverge to infinity at zero velocity, but never mind that.) In practice, using a lower Isp first stage and a higher Isp upper stage partially implements this, and the overall efficiency isn't too bad.