Exactly. The most expensive part of getting to orbit is circularization. It takes a delta-vee of ~9.4km/s to reach LEO from earth. If you could lift yourself into LEO altitude without any thrust, you'd still have to circularize to the orbital velocity of ~8km/s [!]. That's still the majority of your delta-vee!

[!] I am not a rocket scientist, so I haven't taken into account the orbital velocity you'd get from being rooted on the earth at takeoff, but it should be at most ~500m/s.