Wait till we start making LOX (and later, I think, metals) on the Moon from regolith. The process is old and tested on Earth, the missing part is actually experimenting with this on the Moon. And as soon as we have more or less robust Earth-Moon transportation, this will become important, as it will significantly simplify near Moon operations.
Do you know NASA runs, right now, competitions for e.g. lunar ice extraction and purifying?
This is the key state differentiator.
If you bring everything from Earth, it doesn't make sense. (So why are we doing it now?)
But once you start procuring resources from outside gravity well, things look different.
But this isn't necessarily advocating for a full time in-orbit refuelling station either. The plan might be to instead launch 2 or 3 rockets, 1 with the stuff and the others with fuel. They meet up in orbit and the fuel rockets refill the stuff rocket so it can go on to Mars or wherever, then they return to Earth for the next mission.
You don’t get to do this, period. All your crew and all your materials are ultimately coming from Earth (or, in some plans, are mined in-situ). You use considerably more resources getting to your waypoint than you save by leaving from there.
Being farther out of the gravity well isn’t entirely positive either; you’re losing out on significant savings from the Oberth effect. Earth-Moon Lagrange points are well past the minimum-cost orbit.