I guess the idea is that if you can make it to orbit (even if you waste all of your fuel), then you could in principle refuel in-orbit (by getting another unmanned rocket with just fuel into orbit) and thus escape the orbit.

But yes, you still require more energy (exactly sqrt(2) times as much) to "escape" Earth's gravity entirely than to orbit it with a circular orbit.