How much of that material needs to come from Earth, though? If we can get that from the Moon, for example, then you wouldn't need nearly as much fuel to get the raw materials into orbit for manufacturing. Then all you need to do is get the finished product to wherever it needs to go (e.g. in an atmospheric entry vehicle, or to some other orbital factory, or to some far out colony, or what have you).